-- Logs begin at Mon 2022-10-10 16:18:22 UTC, end at Mon 2022-10-10 17:33:32 UTC. -- Oct 10 16:48:53.963349 dv-cinder-834540-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 10 16:48:53.972302 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 10 16:48:53 2022] *** Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: nodename: dv-cinder-834540-2-iscsi Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: machine: x86_64 Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: clock source: unix Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: pcre jit disabled Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: detected number of CPU cores: 4 Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: current working directory: / Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: detected binary path: /usr/bin/uwsgi-core Oct 10 16:48:53.974506 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: your processes number limit is 47680 Oct 10 16:48:53.975147 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: your memory page size is 4096 bytes Oct 10 16:48:53.975147 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: detected max file descriptor number: 2048 Oct 10 16:48:53.975147 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: lock engine: pthread robust mutexes Oct 10 16:48:53.975147 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: thunder lock: enabled Oct 10 16:48:53.975147 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 10 16:48:53.975147 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 10 16:48:54.014062 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: Python main interpreter initialized at 0x55aa14ec39c0 Oct 10 16:48:54.014062 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: python threads support enabled Oct 10 16:48:54.014062 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: your server socket listen backlog is limited to 100 connections Oct 10 16:48:54.014062 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: your mercy for graceful operations on workers is 90 seconds Oct 10 16:48:54.014411 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: mapped 403077 bytes (393 KB) for 2 cores Oct 10 16:48:54.014492 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: *** Operational MODE: preforking *** Oct 10 16:48:54.014751 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: *** uWSGI is running in multiple interpreter mode *** Oct 10 16:48:54.014751 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: spawned uWSGI master process (pid: 81501) Oct 10 16:48:54.015037 dv-cinder-834540-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 10 16:48:54.016430 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: spawned uWSGI worker 1 (pid: 81513, cores: 1) Oct 10 16:48:54.016430 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: spawned uWSGI worker 2 (pid: 81514, cores: 1) Oct 10 16:48:54.016430 dv-cinder-834540-2-iscsi devstack@keystone.service[81501]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 10 16:48:58.345413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 16:48:58.345413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: from cryptography.utils import int_from_bytes Oct 10 16:48:58.346204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 16:48:58.346204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: from cryptography.utils import int_from_bytes Oct 10 16:48:58.364648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 16:48:58.364648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: from cryptography.utils import int_from_bytes Oct 10 16:48:58.365130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 16:48:58.365130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: from cryptography.utils import int_from_bytes Oct 10 16:48:58.395756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 10 16:48:58.395948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 10 16:48:58.396164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] command line args: [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 10 16:48:58.396437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 10 16:48:58.396619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ================================================================================ {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 10 16:48:58.396878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] admin_token = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.397129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] config_dir = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.397319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.397507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] config_source = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.397716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] debug = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.397922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.398430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.398525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.398720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.398969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.399204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.399386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_config_append = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.399565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.399889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_dir = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.400019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_file = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.400190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.400424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.400664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.400913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.401093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.401274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.401521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.401770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.402006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.402244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.402478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.402719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.402969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.403218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.403497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.403992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.404204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] publish_errors = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.404441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.404675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.404857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.405105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.405344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.405582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] standard_threads = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.405817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.406061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.406241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.406418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] use_journal = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.406595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] use_json = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.406770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] use_stderr = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.406949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] use_syslog = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.407181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.407440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.407682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.407943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.408209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.408462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.412732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.413267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.413267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.413472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.413708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.413944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.414184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.414436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.414604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 10 16:48:58.414697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.414797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 10 16:48:58.414909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.414990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] command line args: [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 10 16:48:58.415145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.415246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 10 16:48:58.415380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.415468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ================================================================================ {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 10 16:48:58.415633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.415718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] admin_token = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.415873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.415953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] config_dir = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.416081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.416161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.416264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] config_source = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.416382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.416469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] debug = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.416623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.416708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.416861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.416945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.417029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.417120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.417314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.417393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.417548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.417633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.417782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.417862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.417940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_config_append = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.418018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.418091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.418244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.418331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_dir = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.418405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_file = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.418487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.418642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.418721 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.418886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.418974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.419130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.419208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.419378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.419455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.419535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.419618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.419711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.419846 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.419952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.420107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.420225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.420347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.420457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.420586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.420692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.420825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.420911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.421064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.421148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.421297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.421374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.421546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.421632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.421784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.421861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.422021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.422254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.422340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.422512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] publish_errors = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.422592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.422747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.422825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.422978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.423057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.423187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.423316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.423433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.423513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.423668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.423749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.423902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] standard_threads = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.423980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.424157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.424237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.424403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.424485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.424578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.424656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.424754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] use_journal = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.424878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.424965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] use_json = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.425088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] use_stderr = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.425165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.425262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] use_syslog = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.425341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.425492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 16:48:58.425579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.425747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.425826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.425985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.426970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.427990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.428180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.428275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.428540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.428631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.428707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.429404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.429542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.429657 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.430918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.431943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.432052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.435964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.external = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.password = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] auth.token = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.436975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.437978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.438941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.439905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440259 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.440981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.441937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.442953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.443912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.444090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.444174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.444331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.444415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.444640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.445947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.446950 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.447995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.448982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.449930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.450971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.451863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.453977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.external = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.454947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.password = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] auth.token = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.455920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.456874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.457993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.458933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.459999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.460992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461143 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.461987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.462914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.463856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.464848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.465943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.466978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] role.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.467910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] role.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.468925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.469862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.470954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.471913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.472900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.473847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.474959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.475949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476113 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.476898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.477870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.478979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.479957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.connection = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.480984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481222 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.481958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.482924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.483892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.484996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.485074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81514) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 10 16:48:58.485341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.485613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.485897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.486137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.486370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.486608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.486849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.487098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.487344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.487581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.487828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.488109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.488409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.488654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.488889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.489158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.489392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] role.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.489636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] role.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.489863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.490111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.490356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.490599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.490843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.491088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.491347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.492129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.492377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.492607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.492853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.493098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.493342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.493587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.494023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.494263 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.494542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.494793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.495041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.495288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.495522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.495774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.496044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.496283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.496515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.496747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.498857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.499104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.499341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.499642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.499891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.500153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.500398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.500639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.501161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.501399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.501705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.501994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.502286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.502560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.502901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.503145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.503411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.503780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.connection = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.504111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.504446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.504808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.505102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.505443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.505725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.506069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.506316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.506634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.507010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.507797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.508075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.508318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.508558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.508809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.509041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.509283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.509519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 16:48:58.509690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81513) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 10 16:48:58.517646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.517880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.518111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.526529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.537570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.537799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.538022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.547603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.555078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.555316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.567541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.567769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.611129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.618552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.659738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.659973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.661664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.661900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.748247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.748474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.748686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.748906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.749123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.749401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.787265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.787492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.787711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.787927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.788169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.788940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.789152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.822651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.822888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.823118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.823342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.823564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.823783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.844470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.844697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.844925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.845151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.845372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.845592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.845811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.858274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.873090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.873330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.873555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.873782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.874007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.874230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.874448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81514) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.891356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.891570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 16:48:58.891918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.895512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 16:48:58.895643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.895752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.895752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.895752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.895752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.898070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.898300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.898522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.898744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.898957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.899166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.899373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81513) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 16:48:58.902778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.903058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.903332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.903618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.910101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.910309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 16:48:58.910597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.910681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.910827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 16:48:58.911170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.911535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.911801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.912174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.914067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.914393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.914672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.914955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.915240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.915505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.915776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.916137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.916415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.916692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.916963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.923519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.927116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.927116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.927116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.931767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.934858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.935193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.935480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.935758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.936048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.936320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.936584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.936853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.937119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.937387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.937650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.939896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.941391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.941677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.946892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.947177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.947462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.953816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.954105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.959740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.960060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.960890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.961180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.961462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.961737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.961956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.962051 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.962270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.963413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.963696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.969232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.969516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.969894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.976567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.977734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.978021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.980915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.981207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.981490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.981774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.981855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.982053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.982357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.982628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.982896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.985035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.985337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.989550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.992683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:58.995857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.996181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.996473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.996763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.997048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.997330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.998411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:58.998683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.002812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.006398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.006697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.011451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.013587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.013878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.014152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.014419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.015456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.018667 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.018966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.019246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.019575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.019862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.020161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.021287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.023428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.025000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.025278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.027927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.028243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.028532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.028820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.029207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.038948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.039231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.039498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.041766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.041850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.041850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.041850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.041850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.042230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.042515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.042830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.046943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.048608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.050502 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.050780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.053565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.053846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.054668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.055210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.055597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.058985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.061932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.064861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.066291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.068206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.068554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068616 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.068938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.074812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.076290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.079178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.079577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.079869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.080177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.085090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.088633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.088935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.089017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.089219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.091963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.093479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.095531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.095815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.096100 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81514) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.099002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.099585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81514) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.104342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.105932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.109072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.109464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.109744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.110023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.118488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.118724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.119068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.126790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81513) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 16:48:59.127078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.127355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.127666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.127937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.128244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.128515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.128776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.129060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.129337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.129606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.129876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.130137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.130393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81513) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 16:48:59.134149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aa14ec39c0 pid: 81514 (default app) Oct 10 16:48:59.137436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7d7d3-b026-4e50-965d-7d0d143991c4 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:48:59.137726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7d7d3-b026-4e50-965d-7d0d143991c4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:48:59.137996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7d7d3-b026-4e50-965d-7d0d143991c4 None None] PATH_INFO: `/v3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:48:59.142129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Oct 10 16:48:59 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:48:59.167552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aa14ec39c0 pid: 81513 (default app) Oct 10 16:49:06.788366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:06.788906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:06.788906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:06.790219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 2/2] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:06.798026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:06.806701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:06.806701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:06.870488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: INFO dbcounter [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] Registered counter for database keystone Oct 10 16:49:06.890556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG oslo_db.sqlalchemy.engines [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81513) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 10 16:49:06.896977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writer thread running {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 10 16:49:07.070883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG passlib.handlers.bcrypt [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81513) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 10 16:49:07.071203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG passlib.handlers.bcrypt [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81513) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 10 16:49:07.128683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:07.130279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:07.256974 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1/3] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 10 16:49:07.764689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:07.834044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: INFO dbcounter [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] Registered counter for database keystone Oct 10 16:49:07.855374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG oslo_db.sqlalchemy.engines [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81514) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 10 16:49:07.863197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writer thread running {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 10 16:49:07.980779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed941485-4be4-4cc6-8b05-f4b13b007bfb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:07.984997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:07.985644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:07.985954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:07.986258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:07.994380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:08.046380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 10 16:49:08.046673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877fd1e-0b86-4899-a30d-e3cb59031e5c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:08.056660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 3/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Oct 10 16:49:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 16:49:08.062581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:08.077614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398dae7-98a5-4a24-9478-ddaafcafff3b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:08.081945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:08.083042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:08.083320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:08.083591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] PATH_INFO: `/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:08.085056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3acc8f5bcdf4b42bd810be11a69d5c4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:08.137328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING keystone.common.rbac_enforcer.enforcer [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 10 16:49:08.137665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a9aa5-9a34-4f45-a04e-dfbb86e6f174 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:08.142255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 2/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 10 16:49:08 2022] GET /identity/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:10.250239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0227-a7a4-4d68-9c08-b494ec7acea5 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:10.250846 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0227-a7a4-4d68-9c08-b494ec7acea5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:10.250846 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0227-a7a4-4d68-9c08-b494ec7acea5 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:10.254877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 4/6] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:10.260785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae84f8-6754-4ed4-9825-230408081cff None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:10.261220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae84f8-6754-4ed4-9825-230408081cff None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:10.261334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae84f8-6754-4ed4-9825-230408081cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:10.281553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-beae84f8-6754-4ed4-9825-230408081cff None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:10.283056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-beae84f8-6754-4ed4-9825-230408081cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:10.290712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 3/7] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 16:49:10.554776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0227-a7a4-4d68-9c08-b494ec7acea5 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:10.567019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0227-a7a4-4d68-9c08-b494ec7acea5 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:10.571925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cd488-e703-47ce-a591-7289ae6dc953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:10.572568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cd488-e703-47ce-a591-7289ae6dc953 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:10.572843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cd488-e703-47ce-a591-7289ae6dc953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:10.573111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cd488-e703-47ce-a591-7289ae6dc953 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:10.580586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cd488-e703-47ce-a591-7289ae6dc953 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:10.582325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cd488-e703-47ce-a591-7289ae6dc953 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:10.590812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 5/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Oct 10 16:49:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 16:49:10.595945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae84f8-6754-4ed4-9825-230408081cff None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:10.604817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae84f8-6754-4ed4-9825-230408081cff None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:10.609052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798b44c-57b1-43f2-b24c-41266a2a804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:10.609669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b44c-57b1-43f2-b24c-41266a2a804d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:10.609947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b44c-57b1-43f2-b24c-41266a2a804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:10.610217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b44c-57b1-43f2-b24c-41266a2a804d admin admin] PATH_INFO: `/v3/users/6f35dce3202646389d8bcfb32df2d4b4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:10.611514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798b44c-57b1-43f2-b24c-41266a2a804d admin admin] RBAC: Authorizing `identity:get_user(user_id=6f35dce3202646389d8bcfb32df2d4b4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:10.613343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798b44c-57b1-43f2-b24c-41266a2a804d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:10.614835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 4/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Oct 10 16:49:10 2022] GET /identity/v3/users/6f35dce3202646389d8bcfb32df2d4b4 => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:12.837751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24e782-a540-40c3-a22c-48c11f8a729b None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:12.838263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24e782-a540-40c3-a22c-48c11f8a729b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:12.838352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24e782-a540-40c3-a22c-48c11f8a729b None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:12.839611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 6/10] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:12.843560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72274-4f14-4a62-a01e-593838ee40a2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:12.843999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72274-4f14-4a62-a01e-593838ee40a2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:12.844221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72274-4f14-4a62-a01e-593838ee40a2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:12.844392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:12.844674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:12.844949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:12.845524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 5/11] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:12.853468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:12.853754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:12.854033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:12.873316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:12.874501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:12.887912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG passlib.handlers.bcrypt [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81514) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 10 16:49:12.888326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG passlib.handlers.bcrypt [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81514) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 10 16:49:12.937757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:12.939334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:12.952359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:13.037281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:13.217696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.240621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:13.300172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.321108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40010e3-65d2-4e0b-89c2-9abb10dce7fa None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.354248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.354866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.355144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.355414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.356108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:13.357476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:13.399927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787fab1-35bc-4289-b197-e54778f63e9e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.402253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:13.426142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.sql.core [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 10 16:49:13.426142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 16:49:13.426142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [parameters: {'id': 'ee0a7dfa6af54ece9c71bea91f3ccb8e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 10 16:49:13.426142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81513) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 16:49:13.437372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.438466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.438751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.439023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:49:13.445812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result.read() Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 10 16:49:13.446734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application yield resource Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise self.value Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application session.commit() Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 16:49:13.447917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.session.flush() Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application compat.raise_( Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 16:49:13.449216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 16:49:13.450273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result.read() Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:49:13.451203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [parameters: {'id': 'ee0a7dfa6af54ece9c71bea91f3ccb8e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 10 16:49:13.452352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:13.453670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 7/14] 10.222.0.42 () {60 vars in 1137 bytes} [Mon Oct 10 16:49:13 2022] POST /identity/v3/domains => generated 190 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:13.454844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.456542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:13.465602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:13.466608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:13.467537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:49:13.472706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.538487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:13.552716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510f601-4555-4e8c-846e-5d3a45692a97 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.573139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c719699-b86d-4367-9a87-111bf6b23be6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.587868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.588535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.588818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.589091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.598979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:13.600497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:13.607506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.608151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.608432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.608699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.609364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:13.609662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:13.611097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:49:13.616858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:13.617744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:49:13.618572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:49:13.618572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:49:13.618572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:13.618572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 8/16] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:13.623369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:13.623635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.628798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.656019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:13.720596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858726f-8418-4c74-b1ea-be91dd9fb777 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.727742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba865-ef40-40ea-b167-832572bd939f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.753396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.753996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.754273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.754543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.755183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:13.755480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:13.757014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:13.762342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.762342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.762342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.762342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] PATH_INFO: `/v3/users/6f35dce3202646389d8bcfb32df2d4b4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.763465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] RBAC: Authorizing `identity:get_user(user_id=6f35dce3202646389d8bcfb32df2d4b4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:13.764928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:13.766486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:13 2022] GET /identity/v3/users/6f35dce3202646389d8bcfb32df2d4b4 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:13.771949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:13.801119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:13.809117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:13.872915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:13.879629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc86c1a-36fa-4473-abaf-a2d76bd26e7f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:13.915129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5098e-5366-479a-93c3-db92a17f0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:13.915777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5098e-5366-479a-93c3-db92a17f0e06 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:13.916082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5098e-5366-479a-93c3-db92a17f0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:13.916359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5098e-5366-479a-93c3-db92a17f0e06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:13.917512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5098e-5366-479a-93c3-db92a17f0e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:13.918978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5098e-5366-479a-93c3-db92a17f0e06 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:13.920615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:13.926577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:14.029335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f02f0e-36ac-4b40-937b-c08984cd281d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:14.065021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4ea48-b103-4e64-af23-3f8bc97a15b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:14.065690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ea48-b103-4e64-af23-3f8bc97a15b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:14.065968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ea48-b103-4e64-af23-3f8bc97a15b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:14.066397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ea48-b103-4e64-af23-3f8bc97a15b5 None admin] PATH_INFO: `/v3/domains/default/users/6f35dce3202646389d8bcfb32df2d4b4/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:14.068393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4ea48-b103-4e64-af23-3f8bc97a15b5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6f35dce3202646389d8bcfb32df2d4b4, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:14.070412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4ea48-b103-4e64-af23-3f8bc97a15b5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:14.244986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 10/21] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Oct 10 16:49:13 2022] PUT /identity/v3/domains/default/users/6f35dce3202646389d8bcfb32df2d4b4/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:16.414810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06595f8d-05bc-471d-b6ff-52e3770ddf81 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:16.415372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06595f8d-05bc-471d-b6ff-52e3770ddf81 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:16.415372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06595f8d-05bc-471d-b6ff-52e3770ddf81 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:16.416725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:16.434762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85db9b-16f8-479a-95e4-f63ac62c3e83 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:16.435262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85db9b-16f8-479a-95e4-f63ac62c3e83 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:16.435361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85db9b-16f8-479a-95e4-f63ac62c3e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:16.469401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5d85db9b-16f8-479a-95e4-f63ac62c3e83 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:16.470993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5d85db9b-16f8-479a-95e4-f63ac62c3e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:16.546766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:16.566371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:16.703534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a8a40-00bb-43bf-acaa-e9365f336933 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:16.703738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a8a40-00bb-43bf-acaa-e9365f336933 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:16.704195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a8a40-00bb-43bf-acaa-e9365f336933 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:16.705942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:16.717619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:16.717906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:16.718183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:16.745115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:16.746316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:16.829378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 12/25] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:17.030446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e278b-2c2c-41ae-9be0-d0d201217069 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.030630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e278b-2c2c-41ae-9be0-d0d201217069 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.030963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e278b-2c2c-41ae-9be0-d0d201217069 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.033279 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:17.034468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:17.061803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d59d6a-4427-4211-b886-24d0af218f1c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.062406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d59d6a-4427-4211-b886-24d0af218f1c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.062884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d59d6a-4427-4211-b886-24d0af218f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.093252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-38d59d6a-4427-4211-b886-24d0af218f1c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:17.094813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-38d59d6a-4427-4211-b886-24d0af218f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:17.170140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcf2c-e0b5-4dc2-a714-1beab2b282b0 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:17.195719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:17.207901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:17.208589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.208870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.209144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.216701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 15/27] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:17.218658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94034208-ace2-4543-b14f-136954a4c558 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.221524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94034208-ace2-4543-b14f-136954a4c558 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.221524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94034208-ace2-4543-b14f-136954a4c558 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.221524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 16/28] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:17.222253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94034208-ace2-4543-b14f-136954a4c558 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:17.230218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:17.231928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-fff6e576-e9e3-4657-bf93-3d4cab3fef0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:49:17.263778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:17.264708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:49:17.265472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:49:17.265472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:49:17.265472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:17.265472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 13/29] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:17.266214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7a442-5df8-401f-875d-19305d99eded None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.266500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7a442-5df8-401f-875d-19305d99eded None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.266779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7a442-5df8-401f-875d-19305d99eded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.303382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-83e7a442-5df8-401f-875d-19305d99eded None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:17.308154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-83e7a442-5df8-401f-875d-19305d99eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:17.385303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:17.401870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94034208-ace2-4543-b14f-136954a4c558 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:17.414969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 14/30] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:17.415674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7a442-5df8-401f-875d-19305d99eded None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:17.476748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:17.477363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.477644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.477915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.478744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:17.479971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:17.541063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7a442-5df8-401f-875d-19305d99eded None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:17.544456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:17.576922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:17.577479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.577764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.578044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.578758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:17.579066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:17.580535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:17.595361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:17.600365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:17.706365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcc563-7ca5-450c-ba91-7480f53ea9b1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:17.740843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:17.742070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:17.742356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:17.742631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:17.743358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:17.744802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:17.780082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 17/32] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 10 16:49:17 2022] POST /identity/v3/roles => generated 219 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:17.781058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:17.879205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 16/33] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 10 16:49:17 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:17.880571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:17.936419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:17.968698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea367-188e-40a2-bcd2-0bdce620776d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.035247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356bd5-9cbc-4be6-9343-3e57f3e090c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.036241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356bd5-9cbc-4be6-9343-3e57f3e090c1 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.036713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356bd5-9cbc-4be6-9343-3e57f3e090c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.037134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356bd5-9cbc-4be6-9343-3e57f3e090c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.038592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad356bd5-9cbc-4be6-9343-3e57f3e090c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:18.041820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad356bd5-9cbc-4be6-9343-3e57f3e090c1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:18.098405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4be4d5-d70f-43da-a105-b88cc09ff93c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.137748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.138643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.139124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.139541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.154681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:18.161206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:18.162347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:18.163834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:18.165851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 17/34] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:18.172343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:18.198516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 18/35] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:49:17 2022] POST /identity/v3/projects => generated 301 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:18.204397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b9fd2-7ec1-48a5-b557-e4f181756434 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.204775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b9fd2-7ec1-48a5-b557-e4f181756434 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.205059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b9fd2-7ec1-48a5-b557-e4f181756434 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.212400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 19/36] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:18.219062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.219363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.224713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.230032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:18.251882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:18.253546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:18.282966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:18.304869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a595-5fa7-426c-83af-a301bd697cb7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.344419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 20/37] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:18.345952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.346744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.347215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.347495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.348914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:18.348914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:18.349921 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05dc89-1d92-40ca-9ed8-3e26f7a05e26 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:18.363055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 18/38] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:18.369154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:18.381240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56abc9-64c5-453c-ad7a-a4e0087cec6c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.382356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56abc9-64c5-453c-ad7a-a4e0087cec6c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.382835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56abc9-64c5-453c-ad7a-a4e0087cec6c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.384418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 19/39] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:49:18.394785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.395448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.396256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.422431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:18.424634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:18.511429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 20/40] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:18.512799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7f0f9-814f-40e4-9378-195740d2c107 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.554298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.554947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.555526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.555801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] PATH_INFO: `/v3/users/6f35dce3202646389d8bcfb32df2d4b4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.557068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=6f35dce3202646389d8bcfb32df2d4b4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:18.558533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:18.560158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 21/41] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:18 2022] GET /identity/v3/users/6f35dce3202646389d8bcfb32df2d4b4 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:18.565728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:18.640569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:18.654609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:18.657712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:18.670368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72ae1a-c3db-4e23-8d32-9f5b06b63e91 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.708365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.709404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.710002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.710495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.711818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:18.713652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:18.715666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 21/42] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:18.721867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:18.765199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781f859-fc06-4007-8754-4c7bff9f2fc1 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.798230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.798834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.799112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.799386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.800073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:18.801446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:18.825783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52bc23-83ce-4fd6-ae4d-e807dc13135a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:18.875763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 22/43] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 10 16:49:18 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:18.876709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:18.952006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:18.972924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:18.973670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:18.973955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:18.974234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:18.974993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.domain.id=default, user.id=6f35dce3202646389d8bcfb32df2d4b4) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:18.975308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:18.977525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:18.996062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 22/44] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Oct 10 16:49:18 2022] GET /identity/v3/role_assignments?user.id=6f35dce3202646389d8bcfb32df2d4b4&scope.domain.id=default&role.id=6e700d1e090747d6acdf236f6761cbea => generated 530 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:18.997016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54011b2-5511-469c-bac3-bf62e7c55a28 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:19.036526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b6e48-b65b-417a-a254-9486383383a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:19.037190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6e48-b65b-417a-a254-9486383383a0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:19.037476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6e48-b65b-417a-a254-9486383383a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:19.037753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b6e48-b65b-417a-a254-9486383383a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:19.039518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b6e48-b65b-417a-a254-9486383383a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:19.041068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b6e48-b65b-417a-a254-9486383383a0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:19.042803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:19.048913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:19.053026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:19.161348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b23b-681e-4c2b-9b5a-ae4c15ee0c14 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:19.195834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1782a-5284-4985-ae71-0ec96bde6d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:19.196470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1782a-5284-4985-ae71-0ec96bde6d29 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:19.196749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1782a-5284-4985-ae71-0ec96bde6d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:19.197023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1782a-5284-4985-ae71-0ec96bde6d29 None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:19.197666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1782a-5284-4985-ae71-0ec96bde6d29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:19.199090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1782a-5284-4985-ae71-0ec96bde6d29 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:19.272603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:49:19 2022] POST /identity/v3/projects => generated 312 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:21.406201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15aa11-76c4-4649-8def-a96100424162 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:21.406741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15aa11-76c4-4649-8def-a96100424162 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:21.406741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15aa11-76c4-4649-8def-a96100424162 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:21.408121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:21.413280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:21.413670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:21.413847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:21.431746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:21.432989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:21.481581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:21.497355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:21.764527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15aa11-76c4-4649-8def-a96100424162 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:21.863428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15aa11-76c4-4649-8def-a96100424162 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:21.892703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:21.898285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994f2e-8604-43ce-98d6-b109a30e2a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:21.898913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994f2e-8604-43ce-98d6-b109a30e2a83 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:21.899197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994f2e-8604-43ce-98d6-b109a30e2a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:21.899516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994f2e-8604-43ce-98d6-b109a30e2a83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:21.900673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5994f2e-8604-43ce-98d6-b109a30e2a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:21.902098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5994f2e-8604-43ce-98d6-b109a30e2a83 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:21.903667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:21.909260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:22.004519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007e136-4df7-480d-b4a1-de67208617d3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:22.036304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6809-76c1-49d7-90e2-4fcc3b1b1667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:22.036850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6809-76c1-49d7-90e2-4fcc3b1b1667 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:22.037132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6809-76c1-49d7-90e2-4fcc3b1b1667 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:22.037407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6809-76c1-49d7-90e2-4fcc3b1b1667 None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:22.038599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6809-76c1-49d7-90e2-4fcc3b1b1667 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:22.040015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6809-76c1-49d7-90e2-4fcc3b1b1667 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:22.140938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 25/50] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Oct 10 16:49:21 2022] POST /identity/v3/projects => generated 298 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:24.264487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546db26-3b6d-487b-b395-dec8e3a6e346 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:24.265024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546db26-3b6d-487b-b395-dec8e3a6e346 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:24.265024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546db26-3b6d-487b-b395-dec8e3a6e346 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:24.266401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 26/51] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:24.271454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:24.271877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:24.272197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:24.290476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:24.291652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:24.320546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:24.359468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 26/52] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:24.629080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546db26-3b6d-487b-b395-dec8e3a6e346 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:24.727229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546db26-3b6d-487b-b395-dec8e3a6e346 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:24.760270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2f3e4-d3da-4f8f-90b5-9cf3b08b9d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:24.760893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2f3e4-d3da-4f8f-90b5-9cf3b08b9d5f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:24.761173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2f3e4-d3da-4f8f-90b5-9cf3b08b9d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:24.761445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2f3e4-d3da-4f8f-90b5-9cf3b08b9d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:24.762839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2f3e4-d3da-4f8f-90b5-9cf3b08b9d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:24.764283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2f3e4-d3da-4f8f-90b5-9cf3b08b9d5f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:24.765827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:24.771300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:24.867242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0fecd-cb4c-425a-9bb8-df97b85a553c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:24.900464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8d95e-b28e-475e-ac19-d42b014ded9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:24.901011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8d95e-b28e-475e-ac19-d42b014ded9c None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:24.901293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8d95e-b28e-475e-ac19-d42b014ded9c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:24.901565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8d95e-b28e-475e-ac19-d42b014ded9c None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:24.902222 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8d95e-b28e-475e-ac19-d42b014ded9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:24.903664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8d95e-b28e-475e-ac19-d42b014ded9c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:25.013914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 27/54] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:24 2022] POST /identity/v3/users => generated 275 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:28.265220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c2fb-19be-4103-bbc3-727627639e24 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.265220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c2fb-19be-4103-bbc3-727627639e24 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.265220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c2fb-19be-4103-bbc3-727627639e24 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.266277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 28/55] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:28.270929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8051b2-4ed3-4294-a4f1-9a92bbf35652 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.271829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8051b2-4ed3-4294-a4f1-9a92bbf35652 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.272873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8051b2-4ed3-4294-a4f1-9a92bbf35652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.294576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2c8051b2-4ed3-4294-a4f1-9a92bbf35652 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:28.296094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2c8051b2-4ed3-4294-a4f1-9a92bbf35652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:28.325193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:28.385506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 28/56] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:28.396974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ffd76-f65c-4000-a9e5-c7baa6b7c70c None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.397314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ffd76-f65c-4000-a9e5-c7baa6b7c70c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.397587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ffd76-f65c-4000-a9e5-c7baa6b7c70c None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.398926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 29/57] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:28.403997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56106e-c2ad-4a54-b70f-28032c51fc04 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.404322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56106e-c2ad-4a54-b70f-28032c51fc04 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.404606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56106e-c2ad-4a54-b70f-28032c51fc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.427294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ed56106e-c2ad-4a54-b70f-28032c51fc04 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:28.429019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ed56106e-c2ad-4a54-b70f-28032c51fc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:28.518806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 29/58] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:28.643910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5a798-a5fb-4d66-a6ef-62d9f2836d4a None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.644242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5a798-a5fb-4d66-a6ef-62d9f2836d4a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.644479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5a798-a5fb-4d66-a6ef-62d9f2836d4a None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.646097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 30/59] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:28.651081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010af3-0ff3-4582-8cb1-dbbbe8177841 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.651377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010af3-0ff3-4582-8cb1-dbbbe8177841 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.651655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010af3-0ff3-4582-8cb1-dbbbe8177841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.663175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5a798-a5fb-4d66-a6ef-62d9f2836d4a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:28.679674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6a010af3-0ff3-4582-8cb1-dbbbe8177841 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:28.681574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6a010af3-0ff3-4582-8cb1-dbbbe8177841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:28.772279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 30/60] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:28.773836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d748-c49e-47c7-b68e-5e54c9df2588 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.774347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d748-c49e-47c7-b68e-5e54c9df2588 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.774868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d748-c49e-47c7-b68e-5e54c9df2588 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.776393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 31/61] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:28.780475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:28.781922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.782388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.782840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.795452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5a798-a5fb-4d66-a6ef-62d9f2836d4a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:28.817846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:28.819079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:28.838687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:28.860999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:28.861945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:28.862362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:28.862766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:28.876829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:28.883798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:28.884796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:28.885526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 31/62] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:28.886138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:28.923852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 32/63] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:28.929387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.049568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232599e-c6b9-4da3-b4c1-95b989ec8d22 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.101913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d10ea-2b77-48d2-a5ff-8f72996d5002 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.140408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.141426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.141910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.142501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.143408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:29.143905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.145630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.156574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 33/64] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.157883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.169216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.196226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.205082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ce271-82ed-4fea-a7ca-3d1695a41a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.205720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ce271-82ed-4fea-a7ca-3d1695a41a7b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.206006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ce271-82ed-4fea-a7ca-3d1695a41a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.206284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ce271-82ed-4fea-a7ca-3d1695a41a7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.217738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ce271-82ed-4fea-a7ca-3d1695a41a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-300ce271-82ed-4fea-a7ca-3d1695a41a7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:29.224717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:29.226140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:29.226878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 32/65] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:29.228299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ad23-ea0e-44f4-847e-0fa953595ffb None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.228299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ad23-ea0e-44f4-847e-0fa953595ffb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.228566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ad23-ea0e-44f4-847e-0fa953595ffb None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.229943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 33/66] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:29.231093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05ad23-ea0e-44f4-847e-0fa953595ffb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.276908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6e9c-8f8d-4108-b10a-46a3941310c5 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.311142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.311707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.311989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.312318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.322426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:29.327677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:29.328540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:29.329275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 34/67] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:29.329885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.330619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05ad23-ea0e-44f4-847e-0fa953595ffb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.364670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.365296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.365576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.365877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] PATH_INFO: `/v3/users/54ccb7a8df554b2ca4677075285913c1` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.380523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=54ccb7a8df554b2ca4677075285913c1)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.381977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.388174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 34/68] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/users/54ccb7a8df554b2ca4677075285913c1 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.388174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.392546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.428323 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd0607-4694-4667-8a17-22600628301b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.428975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.461493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d2dc0-7478-4dd9-b2ca-44bd421f37cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.462131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d2dc0-7478-4dd9-b2ca-44bd421f37cf None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.462407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d2dc0-7478-4dd9-b2ca-44bd421f37cf None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.462677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d2dc0-7478-4dd9-b2ca-44bd421f37cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.472430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d2dc0-7478-4dd9-b2ca-44bd421f37cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-f03d2dc0-7478-4dd9-b2ca-44bd421f37cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:29.477617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:29.478527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:29.479435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:29.480637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.480637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.480637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.483195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1cece-053f-4c0d-944b-3d3fe3befaf0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.497133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:29.498291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:29.516098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.516703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.516979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.517246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.517902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:29.518205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.519549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.528016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.528810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.564890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 36/71] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:29.565643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.632688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.633804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b144d7-4210-4dc3-be84-52fcd1500fb7 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.641040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.669583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.670200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.670482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.670758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.671425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:29.671732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.673220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.682808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.683443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d51c2d-6917-42d4-9939-f0686a085a08 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.684015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.722963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.723599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.723998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.724307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.725451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.727014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.733573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.733573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.790370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52b05c-5856-456b-8976-315c08a92e07 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.827046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.827583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.827865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.828180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.828848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:29.829158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.830557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.833229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9dc24-17de-4709-bb96-f689327aa1cb None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.838739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.839540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.856707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.868625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.869272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.869554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.869833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] PATH_INFO: `/v3/users/6f35dce3202646389d8bcfb32df2d4b4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.870982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] RBAC: Authorizing `identity:get_user(user_id=6f35dce3202646389d8bcfb32df2d4b4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.871469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:29.872466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.874125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/users/6f35dce3202646389d8bcfb32df2d4b4 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.874865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:29.940148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326e5d0-28c0-42a6-8d74-2c6532bda51e None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.972832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129344a0-5f40-4a13-a70f-88af4973430a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:29.973122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:29.973755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:29.974036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:29.974315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] PATH_INFO: `/v3/users/54ccb7a8df554b2ca4677075285913c1` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:29.975474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] RBAC: Authorizing `identity:get_user(user_id=54ccb7a8df554b2ca4677075285913c1)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:29.976947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:29.978533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 38/76] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/users/54ccb7a8df554b2ca4677075285913c1 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:29.979295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.010873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.011538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.011818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.012124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/54ccb7a8df554b2ca4677075285913c1/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.014028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] RBAC: Authorizing `identity:create_grant(project_id=72d99d3c78f8497084bf8a42ad9bb84e, user_id=54ccb7a8df554b2ca4677075285913c1, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.015489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.072611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:30.076268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7b482-717a-449d-a130-453313b2e25d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.085311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 39/77] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:29 2022] PUT /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/54ccb7a8df554b2ca4677075285913c1/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:30.086112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.111016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.111863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.112175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.112465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.114108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.116273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.117990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:30.119036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.164922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:30.221420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cefbbc-eeb3-4c3d-a06f-f88a2da3946c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.236940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bc232-2d6a-4e0f-95fd-7288f7714246 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.264001 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.264900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.265456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.265456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] PATH_INFO: `/v3/users/54ccb7a8df554b2ca4677075285913c1` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.266656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=54ccb7a8df554b2ca4677075285913c1)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.268228 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.269995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:30 2022] GET /identity/v3/users/54ccb7a8df554b2ca4677075285913c1 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:30.270704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.282231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.282935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.283226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.283504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.284690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.286147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.288417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:30 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:30.289238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.324379 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:30.384540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b05a8-a03f-4fef-8f18-9776f4ad1fc1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.396243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed8ef6-054e-4bca-aa1b-50a2e7072121 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.404150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:30.419290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.419961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.420270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.420552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.421680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.423278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.424956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:30 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:30.425716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.440863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.441477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.441759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.442031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.442687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.444212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.525835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9996-25e7-4b63-a4df-e8aa1139b5e1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.559725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.560391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.560674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.560952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] PATH_INFO: `/v3/projects/8f8ea29f2ab64d17a88908ded93809ec` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.562081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8ea29f2ab64d17a88908ded93809ec)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.563569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.565187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 42/82] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:30 2022] GET /identity/v3/projects/8f8ea29f2ab64d17a88908ded93809ec => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:30.571817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.572434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 41/83] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:30 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:30.573388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.649465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:30.688190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:30.699027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a6986-7b74-46b7-a093-00c6b494cc99 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.700538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aa3aa-88fa-4c41-ab47-9b560bfa442f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:30.738298 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080767b-df58-45bd-9e08-1e5944e1f768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.739330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080767b-df58-45bd-9e08-1e5944e1f768 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.739772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080767b-df58-45bd-9e08-1e5944e1f768 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.740252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080767b-df58-45bd-9e08-1e5944e1f768 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/6f35dce3202646389d8bcfb32df2d4b4/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.742126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:30.742955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:30.743163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080767b-df58-45bd-9e08-1e5944e1f768 None admin] RBAC: Authorizing `identity:create_grant(project_id=72d99d3c78f8497084bf8a42ad9bb84e, user_id=6f35dce3202646389d8bcfb32df2d4b4, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.743453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:30.743834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/54ccb7a8df554b2ca4677075285913c1/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:30.745205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080767b-df58-45bd-9e08-1e5944e1f768 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.746440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] RBAC: Authorizing `identity:create_grant(project_id=72d99d3c78f8497084bf8a42ad9bb84e, user_id=54ccb7a8df554b2ca4677075285913c1, role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:30.748154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:30.853436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 42/84] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:30 2022] PUT /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/54ccb7a8df554b2ca4677075285913c1/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:30.854089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:30.868050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 43/85] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:30 2022] PUT /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/6f35dce3202646389d8bcfb32df2d4b4/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:30.996100 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ad5ce-ee16-4405-9421-d4e97157f834 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:31.043635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef41c8-530d-46f5-bc00-6c56c61ca6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:31.044371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef41c8-530d-46f5-bc00-6c56c61ca6ab None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:31.044653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef41c8-530d-46f5-bc00-6c56c61ca6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:31.044929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef41c8-530d-46f5-bc00-6c56c61ca6ab None admin] PATH_INFO: `/v3/projects/8f8ea29f2ab64d17a88908ded93809ec/users/54ccb7a8df554b2ca4677075285913c1/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:31.046957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef41c8-530d-46f5-bc00-6c56c61ca6ab None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8ea29f2ab64d17a88908ded93809ec, user_id=54ccb7a8df554b2ca4677075285913c1, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:31.048559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef41c8-530d-46f5-bc00-6c56c61ca6ab None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:31.175469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 43/86] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:30 2022] PUT /identity/v3/projects/8f8ea29f2ab64d17a88908ded93809ec/users/54ccb7a8df554b2ca4677075285913c1/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:32.824165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc660aef-4f42-4107-ad2d-3b4e68320123 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:32.824709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc660aef-4f42-4107-ad2d-3b4e68320123 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:32.824709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc660aef-4f42-4107-ad2d-3b4e68320123 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:32.826261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 44/87] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:32.831523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:32.831916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:32.832101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:32.857060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:32.858415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:32.862836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:32.932419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 44/88] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:33.437987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc660aef-4f42-4107-ad2d-3b4e68320123 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:33.572960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc660aef-4f42-4107-ad2d-3b4e68320123 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:33.591184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:33.613777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b31f1-075f-4e22-a31c-f0dfeb563ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:33.614424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b31f1-075f-4e22-a31c-f0dfeb563ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:33.618306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b31f1-075f-4e22-a31c-f0dfeb563ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:33.618306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b31f1-075f-4e22-a31c-f0dfeb563ea4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:33.628752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b31f1-075f-4e22-a31c-f0dfeb563ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-d32b31f1-075f-4e22-a31c-f0dfeb563ea4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:33.634508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:33.635560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:33.636529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 45/89] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:33.645397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:33.760176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736af14f-c477-43d6-896b-b655320a4b7f None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:33.760390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736af14f-c477-43d6-896b-b655320a4b7f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:33.760675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736af14f-c477-43d6-896b-b655320a4b7f None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:33.762599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 46/90] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:49:33.767611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61486131-ec59-408f-9abb-661a8f16b53f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:33.767895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61486131-ec59-408f-9abb-661a8f16b53f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:33.768201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61486131-ec59-408f-9abb-661a8f16b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:33.836477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-61486131-ec59-408f-9abb-661a8f16b53f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:33.837817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-61486131-ec59-408f-9abb-661a8f16b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:33.867406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf019f-091c-4adc-b6c9-7d2af2bd8453 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:33.925854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:33.927191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:33.927935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:33.928946 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:33.929800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:33.930508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:33.932103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:33.948949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 45/91] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:33.957288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:33.962581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.050230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 47/92] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 10 16:49:34.051630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff152b-dc70-4bb7-aca0-b0efaa20b9ba None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.051922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff152b-dc70-4bb7-aca0-b0efaa20b9ba None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.052230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff152b-dc70-4bb7-aca0-b0efaa20b9ba None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.054156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 48/93] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:34.065390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989b47-834b-41aa-b06e-f543aeab87ce None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.065672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989b47-834b-41aa-b06e-f543aeab87ce None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.065950 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989b47-834b-41aa-b06e-f543aeab87ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.075324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87322997-db5d-4321-a395-c3056178e9c5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:34.093971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-de989b47-834b-41aa-b06e-f543aeab87ce None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:34.095220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-de989b47-834b-41aa-b06e-f543aeab87ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:34.117005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd8f22-c031-4521-b898-8639d1705704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:34.117648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd8f22-c031-4521-b898-8639d1705704 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.117930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd8f22-c031-4521-b898-8639d1705704 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.118205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd8f22-c031-4521-b898-8639d1705704 None admin] PATH_INFO: `/v3/users/54ccb7a8df554b2ca4677075285913c1` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.120085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd8f22-c031-4521-b898-8639d1705704 None admin] RBAC: Authorizing `identity:get_user(user_id=54ccb7a8df554b2ca4677075285913c1)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:34.121540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd8f22-c031-4521-b898-8639d1705704 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:34.127838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 46/94] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:33 2022] GET /identity/v3/users/54ccb7a8df554b2ca4677075285913c1 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:34.129023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181685-5ad6-4197-b040-e0351f3521cc None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.129316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181685-5ad6-4197-b040-e0351f3521cc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.129598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181685-5ad6-4197-b040-e0351f3521cc None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.131178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 47/95] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:34.132141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76201bf3-fd88-44f2-9069-35494997614c None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.136338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76201bf3-fd88-44f2-9069-35494997614c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.136338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76201bf3-fd88-44f2-9069-35494997614c None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.136338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 48/96] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:34.136338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76201bf3-fd88-44f2-9069-35494997614c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:34.198640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 49/97] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:34.200378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4458a-a251-419a-868c-d7a9970c56a8 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.200899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4458a-a251-419a-868c-d7a9970c56a8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.201375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4458a-a251-419a-868c-d7a9970c56a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.221159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6ec4458a-a251-419a-868c-d7a9970c56a8 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:34.222822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6ec4458a-a251-419a-868c-d7a9970c56a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:34.245689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.258610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.307503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 50/98] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:34.309230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.309832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.310288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.325341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76201bf3-fd88-44f2-9069-35494997614c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:34.330462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:34.332740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:34.414406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 51/99] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:34.416125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:34.416981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.417506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.417978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.423946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:34.540147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.547379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb70a-b86e-482d-bf27-38c7648286a4 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:34.587380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:34.587573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:34.588492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.588832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:34.589199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.589684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.602847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:34.609048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:34.609904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:34.610649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:34.613835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 52/100] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:34.621229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:34.708624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.731007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe2d39-98b1-4cc5-9fc3-e1507858afd4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:34.743235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 49/101] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:34.748941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:34.768391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:34.769013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.769299 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.769579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.770250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:34.770558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:34.772507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:34.780965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 53/102] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:34.781759 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:34.803830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.858127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b387c-b80a-44ed-8998-bfbf2a289306 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:34.888570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c486af-3804-4383-b95c-0e4e73fc67ba None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:34.893134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:34.893766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.894042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.894311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.895411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:34.896904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:34.899254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 50/103] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:34.899838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:34.926617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0f79e-75ba-4809-a48b-7a9c2f18e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:34.927256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0f79e-75ba-4809-a48b-7a9c2f18e9fb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.927541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0f79e-75ba-4809-a48b-7a9c2f18e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.927819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0f79e-75ba-4809-a48b-7a9c2f18e9fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.938445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0f79e-75ba-4809-a48b-7a9c2f18e9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:34.943108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1de0f79e-75ba-4809-a48b-7a9c2f18e9fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:34.943651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:34.945076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:34.945825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 54/104] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:34.946435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54511a46-9d57-4608-8c77-7720a31b3e8f None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:34.947012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54511a46-9d57-4608-8c77-7720a31b3e8f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:34.947012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54511a46-9d57-4608-8c77-7720a31b3e8f None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:34.948331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 55/105] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:34.952813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54511a46-9d57-4608-8c77-7720a31b3e8f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.005014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287198f-72ff-45c6-a3b0-cdfa0d5aa002 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.030823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.037989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.038597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.038877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.039148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.040287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.048415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.048415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 51/106] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.048415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.052942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54511a46-9d57-4608-8c77-7720a31b3e8f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.086716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.087349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.087636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.087920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] PATH_INFO: `/v3/users/dd84048d40ac4c558ddde08516be1686` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.101450 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] RBAC: Authorizing `identity:get_user(user_id=dd84048d40ac4c558ddde08516be1686)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.102908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.104563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 56/107] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:34 2022] GET /identity/v3/users/dd84048d40ac4c558ddde08516be1686 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.109708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.145662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f487c-d9ad-4fdc-b0b4-02a9870d5b3d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.152474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.178070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.179245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.179532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.179808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.190548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:35.196055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:35.196930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:35.197671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 52/108] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:35.200833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.207371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14ae0-7e74-482c-93a7-24ebde98a7ae None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.241639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a36d7-3e26-49ba-ae74-d498a183a49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.242247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a36d7-3e26-49ba-ae74-d498a183a49a None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.242534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a36d7-3e26-49ba-ae74-d498a183a49a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.242805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a36d7-3e26-49ba-ae74-d498a183a49a None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.243447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a36d7-3e26-49ba-ae74-d498a183a49a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.244932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a36d7-3e26-49ba-ae74-d498a183a49a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.250211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.295945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5f7e-cc9d-4c95-a91b-8fbf046ea4d0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.327082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 57/109] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:49:35 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:35.328453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.328746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.329021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.335229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.335897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.336214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.336487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.337184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:35.337491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.338935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.348366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 53/110] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.353782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.353923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:35.353923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:35.382769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.427641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 58/111] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:35.441145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.480658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aa205-e6d0-4770-8cff-e7573936e2c3 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.526608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.527399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.527784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.528869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.528869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] RBAC: Adding query filter params (role.id=3a658e34b7d14750a0169f576d87baef, scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e, user.id=54ccb7a8df554b2ca4677075285913c1) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:35.529138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.530568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.540219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 54/112] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/role_assignments?user.id=54ccb7a8df554b2ca4677075285913c1&scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e&role.id=3a658e34b7d14750a0169f576d87baef => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.540975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.581311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c2535-4387-4dee-b66e-7bdaa18be019 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.602989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.625673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.626332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.626614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.626887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.628227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.629862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.632046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 59/113] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.637372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.672052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.694141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d3800-ed23-400b-b50b-d6a6ff4e079a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.731530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.732189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.732474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.732750 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.733429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:35.733738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.735141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.745547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 55/114] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.749476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.755295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07028c32-7a48-4e1e-93b6-97ebd3954f05 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.807437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.808127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.808570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.808877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] PATH_INFO: `/v3/users/54ccb7a8df554b2ca4677075285913c1` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.810075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] RBAC: Authorizing `identity:get_user(user_id=54ccb7a8df554b2ca4677075285913c1)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.811510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.813958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 60/115] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/users/54ccb7a8df554b2ca4677075285913c1 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.814576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.879537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7edece-58fb-4a21-847e-c78c4252b9e5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.884454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:35.916457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.917131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.917411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.917688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/dd84048d40ac4c558ddde08516be1686/roles/7a8d99c2f3c1479fad1ceeae9a792694` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.919706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] RBAC: Authorizing `identity:create_grant(project_id=72d99d3c78f8497084bf8a42ad9bb84e, user_id=dd84048d40ac4c558ddde08516be1686, role_id=7a8d99c2f3c1479fad1ceeae9a792694)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.921191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.933025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d08be-52e0-4fb5-97f9-2453239735b4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:35.966794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:35.967424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:35.967713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:35.967984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] PATH_INFO: `/v3/users/6f35dce3202646389d8bcfb32df2d4b4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:35.969129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] RBAC: Authorizing `identity:get_user(user_id=6f35dce3202646389d8bcfb32df2d4b4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:35.970545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:35.972148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 61/116] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/users/6f35dce3202646389d8bcfb32df2d4b4 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:35.972907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:35.989287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 56/117] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:35 2022] PUT /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/users/dd84048d40ac4c558ddde08516be1686/roles/7a8d99c2f3c1479fad1ceeae9a792694 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:35.990124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.017182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.104161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e98b41-0bb9-419b-9522-28b2f2f2631b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.132194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.150138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.150734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.151025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.151299 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.153161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0f9a6-1ba0-47ec-9ea8-8cebebed7144 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.164613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:36.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:36.170655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:36.171557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:36.172196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 62/118] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:36.172670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.209231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.216063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.216063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.216063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.311991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.313830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.315564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c458e29-3194-4619-9a24-908495ed5e88 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.330625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.350842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.351463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.352059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.352059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.404481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.424248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.425743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.432101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 57/119] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:36.432971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.538451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 63/120] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:36.543674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.550373 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac8870-2981-41d4-8ec2-8e6302293c7c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.567181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.588416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.589165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.589520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.589868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.590609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:36.590986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.592496 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.602762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 58/121] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:36.603605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.665022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.691230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79065b-76c2-46b5-857c-81c19d658d08 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.712442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1b532-eb91-4f3a-a846-a475ac28bb7c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.743530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.744727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.745154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.745765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.747684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.749359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.751310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 64/122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:36.751422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.752385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.757391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.757833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.758232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.760098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.762730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.764598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 59/123] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:36.765511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.821528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:36.881460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c5800-eace-4181-8090-b8ea7f85af16 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.892682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86791ab-8b26-44a7-a4ba-3f3d16736ec0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:36.929878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.931047 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.931163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:36.931689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:36.931973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.932259 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:36.932366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.932765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] PATH_INFO: `/v3/users/54ccb7a8df554b2ca4677075285913c1` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:36.932997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] RBAC: Adding query filter params (role.id=215077d32ef84d5eaa4d487616aa2ec3, scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e, user.id=54ccb7a8df554b2ca4677075285913c1) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:36.933309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.934451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] RBAC: Authorizing `identity:get_user(user_id=54ccb7a8df554b2ca4677075285913c1)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:36.934987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.936523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:36.939282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 65/124] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/users/54ccb7a8df554b2ca4677075285913c1 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:36.940505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.942190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 60/125] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/role_assignments?user.id=54ccb7a8df554b2ca4677075285913c1&scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e&role.id=215077d32ef84d5eaa4d487616aa2ec3 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:36.949313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:36.965873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:37.067896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3218d82-b986-4eb5-9217-3baff6b3502d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:37.111525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:37.112498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:37.113300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:37.113779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:37.115058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e, user.id=6f35dce3202646389d8bcfb32df2d4b4) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:37.115600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:37.117295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:37.125141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 66/126] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/role_assignments?user.id=6f35dce3202646389d8bcfb32df2d4b4&scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:37.172119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd25191-dcc2-41c0-abe2-5bce2e7b71d5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:37.183942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:37.210090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:37.210650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:37.210934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:37.211211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:37.287886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:37.289415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:37.396208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 61/127] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:37.401968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:37.488694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:37.507517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b61ac-ec40-4fe4-9814-d781023141e9 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:37.540382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e56162-59ef-46e3-a315-6749f51194c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:37.541013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56162-59ef-46e3-a315-6749f51194c8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:37.541293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56162-59ef-46e3-a315-6749f51194c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:37.541561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56162-59ef-46e3-a315-6749f51194c8 None admin] PATH_INFO: `/v3/projects/8f8ea29f2ab64d17a88908ded93809ec` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:37.542671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e56162-59ef-46e3-a315-6749f51194c8 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8ea29f2ab64d17a88908ded93809ec)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:37.544124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e56162-59ef-46e3-a315-6749f51194c8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:37.545913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 67/128] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:37 2022] GET /identity/v3/projects/8f8ea29f2ab64d17a88908ded93809ec => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:37.551503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:37.562631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:37.653497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161c2c3-e9ba-4e25-beb5-7582a5c336ff None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:37.690502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:37.691177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:37.691527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:37.691809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:37.692587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] RBAC: Adding query filter params (role.id=3a658e34b7d14750a0169f576d87baef, scope.project.id=8f8ea29f2ab64d17a88908ded93809ec, user.id=54ccb7a8df554b2ca4677075285913c1) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:37.692899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:37.694941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107084-eff0-4453-8a6f-df4f093c56b1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:37.700838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 62/129] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:37 2022] GET /identity/v3/role_assignments?user.id=54ccb7a8df554b2ca4677075285913c1&scope.project.id=8f8ea29f2ab64d17a88908ded93809ec&role.id=3a658e34b7d14750a0169f576d87baef => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:37.771501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e023-ad17-4793-851d-735d809476e2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:37.771772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e023-ad17-4793-851d-735d809476e2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:37.772012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e023-ad17-4793-851d-735d809476e2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:37.773633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 68/130] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:37.779340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6e13-6c6d-47c9-9548-1489b796d796 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:37.779630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6e13-6c6d-47c9-9548-1489b796d796 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:37.779917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6e13-6c6d-47c9-9548-1489b796d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:37.799390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-44ab6e13-6c6d-47c9-9548-1489b796d796 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:37.800635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-44ab6e13-6c6d-47c9-9548-1489b796d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:37.876991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 63/131] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:38.144662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e023-ad17-4793-851d-735d809476e2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:38.241675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2876b-c008-4e8b-b9e2-5ec312b009f6 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:38.242219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2876b-c008-4e8b-b9e2-5ec312b009f6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:38.242219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2876b-c008-4e8b-b9e2-5ec312b009f6 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:38.243558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 64/132] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:49:38.245917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e023-ad17-4793-851d-735d809476e2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:38.248409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:38.248684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:38.248956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:38.255073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:38.267368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:38.268577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:38.280481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:38.281122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:38.281407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:38.281683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:38.282798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:38.284265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:38.286016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 69/133] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:38.291786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:38.311012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:38.339295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 65/134] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:38.394223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3058f-f9f6-4c49-8ebb-4fb5b2dd1039 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:38.427314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:38.427929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:38.428257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:38.428533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:38.429177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:38.430877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:38.505307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 70/135] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:38 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:38.611680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:38.711530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21011-5c6e-4f62-9d81-89f6af66eccf None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:38.740166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:38.745291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:38.745971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:38.746256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:38.746542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:38.757269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:38.762096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:38.764915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:38.765668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 66/136] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:38.769127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:38.852510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:38.879385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74886e6-6e77-4b9b-852a-dd9ba97acf66 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:38.915888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:38.916548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:38.916845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:38.917118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:38.917794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:38.918097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:38.919479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:38.929215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 71/137] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:38.935467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:39.040759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e30cd-bb7e-4aad-a6bc-e259f5d39912 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:39.076992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:39.077631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:39.077912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:39.078184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] PATH_INFO: `/v3/users/dd84048d40ac4c558ddde08516be1686` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:39.079333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] RBAC: Authorizing `identity:get_user(user_id=dd84048d40ac4c558ddde08516be1686)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:39.080799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:39.082455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 67/138] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:38 2022] GET /identity/v3/users/dd84048d40ac4c558ddde08516be1686 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:39.089123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:39.195190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a10d9-4ba4-4149-a41d-f416eb73460b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:39.231051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:39.231984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:39.232486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:39.232933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:39.236668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:39.302990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:39.305681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:39.404765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 72/139] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:39.410654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:39.472836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:39.517148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92a6f6-3907-4acd-9ea3-74ed087ff597 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:39.551872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4c8da-6d27-47b3-9724-75bc2baf4107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:39.552544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4c8da-6d27-47b3-9724-75bc2baf4107 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:39.552830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4c8da-6d27-47b3-9724-75bc2baf4107 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:39.553721 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4c8da-6d27-47b3-9724-75bc2baf4107 None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:39.554816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4c8da-6d27-47b3-9724-75bc2baf4107 None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:39.556555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4c8da-6d27-47b3-9724-75bc2baf4107 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:39.558377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 68/140] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:39 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:39.564283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:39.599742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:39.669119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ddfa8-c0a2-48a4-814f-dfdbe6272214 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:39.704158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:39.704772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:39.705102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:39.705380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:39.706110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] RBAC: Adding query filter params (role.id=7a8d99c2f3c1479fad1ceeae9a792694, scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e, user.id=dd84048d40ac4c558ddde08516be1686) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:39.706420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:39.707846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b293456-5bb4-4d9c-85ac-02093834f8d2 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:39.714465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 73/141] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:39 2022] GET /identity/v3/role_assignments?user.id=dd84048d40ac4c558ddde08516be1686&scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e&role.id=7a8d99c2f3c1479fad1ceeae9a792694 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:40.779558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4a3e1-c5b3-4b78-815a-182b0e2d6e6e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:40.780225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4a3e1-c5b3-4b78-815a-182b0e2d6e6e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:40.780225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4a3e1-c5b3-4b78-815a-182b0e2d6e6e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:40.781653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 69/142] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:40.786592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223aa9-06f3-4201-87e6-20bf3e9ac21c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:40.786984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223aa9-06f3-4201-87e6-20bf3e9ac21c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:40.787170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223aa9-06f3-4201-87e6-20bf3e9ac21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:40.808291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-18223aa9-06f3-4201-87e6-20bf3e9ac21c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:40.809511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-18223aa9-06f3-4201-87e6-20bf3e9ac21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:40.886921 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 74/143] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:40.916585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04801cef-d7e0-4c4f-a690-15e181ba1642 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:40.916934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04801cef-d7e0-4c4f-a690-15e181ba1642 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:40.917278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04801cef-d7e0-4c4f-a690-15e181ba1642 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:40.918983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 70/144] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:40.924270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991b52-2ca2-43e3-805f-928b1e044dff None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:40.924562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991b52-2ca2-43e3-805f-928b1e044dff None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:40.924833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991b52-2ca2-43e3-805f-928b1e044dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:40.943819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f3991b52-2ca2-43e3-805f-928b1e044dff None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:40.945007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f3991b52-2ca2-43e3-805f-928b1e044dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:40.969033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.018219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 75/145] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:41.115082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b69c2-9da6-4f03-bb3d-645b9d1b8497 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.115278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b69c2-9da6-4f03-bb3d-645b9d1b8497 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.115594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b69c2-9da6-4f03-bb3d-645b9d1b8497 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.117012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 71/146] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:41.122124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.122415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.122685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.140510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:41.141687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:41.162591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b69c2-9da6-4f03-bb3d-645b9d1b8497 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.211637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 76/147] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:41.265467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b69c2-9da6-4f03-bb3d-645b9d1b8497 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.292125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.294430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.300429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.306067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.306067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.306067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.312046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:41.316829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:41.317866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:41.318855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 72/148] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:49:41.323672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.390333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.400962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42ac1e-a973-4b3e-8c00-5e53974ca205 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.428617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a757f4-dc30-490c-956f-aad6991d9123 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.437149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.437906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.438191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.438464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.449782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:41.455078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:41.455908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:41.457415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 77/149] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:49:41.462566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.465788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.466474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.466764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.467070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.467749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:41.468068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.469452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.477817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 73/150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:41.483281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.531866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.560413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cb08f-a15f-4f35-b4f6-aabc7c161fe0 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.580375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aed74b-25a0-45b3-a495-7d1239b8b67f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.592814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.593419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.593692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.593962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.594609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:41.594908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.596297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.603854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 78/151] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:41.604655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.613391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.614018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.614300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.614577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] PATH_INFO: `/v3/users/5ac3efa5add049a0a3f03629c001e75d` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.620380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.627544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=5ac3efa5add049a0a3f03629c001e75d)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.635679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.635679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 74/152] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/users/5ac3efa5add049a0a3f03629c001e75d => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:41.635679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.705168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95bdd-9694-4449-963a-af02e743fcfc None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.726618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47be5-911a-4a48-96ae-dea20d3c3ca3 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.735731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.737904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.738566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.738844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.739116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.740310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.748106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.748106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 79/153] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:41.748106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.760442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.761061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.761337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.761608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] PATH_INFO: `/v3/users/5ac3efa5add049a0a3f03629c001e75d` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.762709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] RBAC: Authorizing `identity:get_user(user_id=5ac3efa5add049a0a3f03629c001e75d)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.764164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.770039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 75/154] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/users/5ac3efa5add049a0a3f03629c001e75d => generated 283 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:41.770039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.861449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa895e6c-9582-4594-bd97-a420ebfa05c7 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.866190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:41.887514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695991f-a14f-4dc7-a157-1143a938436b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:41.894863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.895463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.895739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.896080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.896752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.898167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.919933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:41.923369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:41.923369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:41.923369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:41.923369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:41.923750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:41.925278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 80/155] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:41.926032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:41.982534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 76/156] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:41 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:41.983510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:42.029963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da4f47-e72d-4c0f-b923-0dcdf306cd0a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:42.039685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:42.069955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:42.070609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:42.070973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:42.071162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:42.072446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:42.073928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:42.075830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 81/157] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:41 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:42.081675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:42.101582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:42.120178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def6bc5-85fe-4402-a510-a7ad33d38d65 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:42.159934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094bfd-01eb-4f8a-8bc4-cc2979b209e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:42.159934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094bfd-01eb-4f8a-8bc4-cc2979b209e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:42.160400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094bfd-01eb-4f8a-8bc4-cc2979b209e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:42.160567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094bfd-01eb-4f8a-8bc4-cc2979b209e7 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/5ac3efa5add049a0a3f03629c001e75d/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:42.162540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75094bfd-01eb-4f8a-8bc4-cc2979b209e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6745b8eec4c16b05969f452c0ebe6, user_id=5ac3efa5add049a0a3f03629c001e75d, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:42.164177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75094bfd-01eb-4f8a-8bc4-cc2979b209e7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:42.192168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639105b-1b7a-42d7-8a14-cd2068e857e4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:42.226616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411d33f-4c36-4955-839f-e6366948e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:42.227274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d33f-4c36-4955-839f-e6366948e8ca None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:42.227550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d33f-4c36-4955-839f-e6366948e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:42.227820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d33f-4c36-4955-839f-e6366948e8ca None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/5ac3efa5add049a0a3f03629c001e75d/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:42.230023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411d33f-4c36-4955-839f-e6366948e8ca None admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6745b8eec4c16b05969f452c0ebe6, user_id=5ac3efa5add049a0a3f03629c001e75d, role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:42.231445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411d33f-4c36-4955-839f-e6366948e8ca None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:42.240123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 77/158] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:41 2022] PUT /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/5ac3efa5add049a0a3f03629c001e75d/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:42.315661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 82/159] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:42 2022] PUT /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/5ac3efa5add049a0a3f03629c001e75d/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:49:44.257835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df61c3-4046-4ee3-92a4-ddac5d77409f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.258347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df61c3-4046-4ee3-92a4-ddac5d77409f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.258445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df61c3-4046-4ee3-92a4-ddac5d77409f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.260014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 78/160] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:44.265115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e36f71-37a6-4d16-8caa-fbd682a9a2a5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.265517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e36f71-37a6-4d16-8caa-fbd682a9a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.265680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e36f71-37a6-4d16-8caa-fbd682a9a2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.285248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a4e36f71-37a6-4d16-8caa-fbd682a9a2a5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:44.286433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a4e36f71-37a6-4d16-8caa-fbd682a9a2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:44.323506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae552c5-bf33-43cc-b8a6-b27075a6cd04 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.323839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae552c5-bf33-43cc-b8a6-b27075a6cd04 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.324109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae552c5-bf33-43cc-b8a6-b27075a6cd04 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.325554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 79/161] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:49:44.330278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233b49-9f0c-4aca-ae7f-0ed268590c5a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.330559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233b49-9f0c-4aca-ae7f-0ed268590c5a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.330812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233b49-9f0c-4aca-ae7f-0ed268590c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.350761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d0233b49-9f0c-4aca-ae7f-0ed268590c5a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:44.351966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d0233b49-9f0c-4aca-ae7f-0ed268590c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:44.362290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 83/162] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:44.428173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 80/163] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:44.483943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7462ba-0c54-45d9-a7b0-3f0445b5105e None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.484311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7462ba-0c54-45d9-a7b0-3f0445b5105e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.484509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7462ba-0c54-45d9-a7b0-3f0445b5105e None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.486117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 84/164] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:44.491168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.491459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.491735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.507188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934925b-c060-4ea4-affc-18aa15513736 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.507471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934925b-c060-4ea4-affc-18aa15513736 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.507741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934925b-c060-4ea4-affc-18aa15513736 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.509314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 85/165] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:49:44.512575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:44.513951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:44.514284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.514571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.514844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.528785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:44.532578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:44.535543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:44.536983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:44.583293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 81/166] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:44.608895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 86/167] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:44.654527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:44.705280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:44.771113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591b3fc-ef9d-45cf-9670-9b23a6e8b06d None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:44.817767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:44.818402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.819235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.819510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.823873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53889da-f938-4c17-a41b-4f249c6e5f98 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:44.830565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:44.836007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:44.836959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:44.837743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 82/168] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:44.844192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:44.860082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:44.867252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:44.867902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.871768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.871768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.871768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:44.871768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:44.876132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 87/169] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:44.876132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:44.879987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:44.953197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762d878-17d6-4e5c-88fb-3a97ce459137 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:44.984300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d897-0d5e-40c0-83f0-78b16d7c3838 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:44.986398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:44.987004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:44.987281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:44.987552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:44.988242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:44.988546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:44.989923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:44.998357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 83/170] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:44.999124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.017494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.018125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.018399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.018665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.028898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:45.034149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:45.035029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 88/171] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:45.041085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.075066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.097661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.098099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa154e5-98c6-4ca2-9bcc-4c8a0fec457b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.130717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.131888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.132190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.132458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.133125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.134570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afc86-2a48-4e0e-a87f-ffbf46d99d88 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.134676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.167268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.167968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.168268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.168537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.178757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:45.184301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:45.185360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:45.186152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 89/172] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:45.186794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.204086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 84/173] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:44 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:49:45.204086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.298852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58619b-701d-495c-9bd6-ceaf20babf62 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.313098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5efcf-f0d3-4369-ac52-8002e86a804f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.325618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.337052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.337715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.338012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.338275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] PATH_INFO: `/v3/users/1b54bdb0a2854da6b4f1cd52fcf3ea61` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.347776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.353469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.353655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] RBAC: Authorizing `identity:get_user(user_id=1b54bdb0a2854da6b4f1cd52fcf3ea61)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.354072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.354352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.354618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.355002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.355291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:45.355590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.356981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.357765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 90/174] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/users/1b54bdb0a2854da6b4f1cd52fcf3ea61 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:45.358588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.371656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 85/175] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:45.374277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.482758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b1e6c-cd03-451f-ba27-3bb22939bcf8 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.486683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04beef0a-6b0b-4fb2-8704-3574cc2243bb None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.533486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.534108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.534391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.534663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.535341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:45.535644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.537050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.538027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.538869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.539308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.539716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.541093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.542669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.544671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 86/176] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:45.545592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.547135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 91/177] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:45.552748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.612454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.619419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.659985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0be21c-14c4-476e-8d63-0675dadc6661 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.693185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70df6f4-9cba-4f49-823e-d8400d28d1a9 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.696922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.700311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.700311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.700311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] PATH_INFO: `/v3/users/5ac3efa5add049a0a3f03629c001e75d` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.700311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] RBAC: Authorizing `identity:get_user(user_id=5ac3efa5add049a0a3f03629c001e75d)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.700736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.702349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 87/178] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/users/5ac3efa5add049a0a3f03629c001e75d => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:45.703024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.741120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.742696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.743127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.743559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/1b54bdb0a2854da6b4f1cd52fcf3ea61/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.746740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6745b8eec4c16b05969f452c0ebe6, user_id=1b54bdb0a2854da6b4f1cd52fcf3ea61, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.748396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.815294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc0b87-60fe-4c70-941a-b80692ed82af None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.823964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 92/179] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:45 2022] PUT /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/1b54bdb0a2854da6b4f1cd52fcf3ea61/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:45.825129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.852583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:45.874965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:45.875416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:45.875699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:45.875970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] PATH_INFO: `/v3/users/5ac3efa5add049a0a3f03629c001e75d` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:45.878551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] RBAC: Authorizing `identity:get_user(user_id=5ac3efa5add049a0a3f03629c001e75d)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:45.882487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:45.885298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 88/180] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/users/5ac3efa5add049a0a3f03629c001e75d => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:45.893845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:45.980827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b57df-672c-4909-aef5-9b99847895eb None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:45.983941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:46.024954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:46.025592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:46.025870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:46.026139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:46.065550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f30f2-8a8d-4311-9ee6-ce19f0f7ff24 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:46.111883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:46.113033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:46.113453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:46.113678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:46.114076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:46.115486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:46.237521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 93/181] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:46.243399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:46.244533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:46.247046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:46.267644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:46.269388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:46.354327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16a985-c40b-4343-9f91-b9c8929ee003 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:46.394345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:46.395005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:46.395333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:46.395609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:46.396944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:46.398816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:46.400624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 94/182] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:46 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:46.431905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:46.480057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 89/183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:46.485479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:46.525776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:46.570785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13afafb-ed57-4d66-8b8d-a27f92dd462f None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:46.632110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:46.632967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:46.633392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:46.633795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:46.634657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6, user.id=5ac3efa5add049a0a3f03629c001e75d) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:46.635103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:46.637596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:46.649358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 95/184] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:46 2022] GET /identity/v3/role_assignments?user.id=5ac3efa5add049a0a3f03629c001e75d&scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:46.672555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:46.680019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea5c8e-4f4c-427f-aca6-5ad6c99f29f2 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:46.717925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5efbf-3b76-40ff-803b-73329423e6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:46.718635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5efbf-3b76-40ff-803b-73329423e6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:46.718921 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5efbf-3b76-40ff-803b-73329423e6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:46.719186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5efbf-3b76-40ff-803b-73329423e6f7 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:46.720428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5efbf-3b76-40ff-803b-73329423e6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:46.721898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5efbf-3b76-40ff-803b-73329423e6f7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:46.723624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 90/185] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:46 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:46.730490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:46.854194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d393-a4cf-4a93-bedc-647a4639930c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:46.905251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:46.905788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:46.906071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:46.906346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:46.907074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] RBAC: Adding query filter params (role.id=215077d32ef84d5eaa4d487616aa2ec3, scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6, user.id=5ac3efa5add049a0a3f03629c001e75d) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:46.907382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:46.908848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71cd54-419a-4546-8208-adddfa2687dc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:46.911770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:46.915593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 96/186] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:46 2022] GET /identity/v3/role_assignments?user.id=5ac3efa5add049a0a3f03629c001e75d&scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6&role.id=215077d32ef84d5eaa4d487616aa2ec3 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:47.596687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af383cc9-ff94-4e16-8683-8bc5b7dbc95c None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:47.597230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af383cc9-ff94-4e16-8683-8bc5b7dbc95c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:47.597230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af383cc9-ff94-4e16-8683-8bc5b7dbc95c None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:47.598870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 91/187] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:47.603718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bee25-4ce0-4e69-8c8c-6552f73d0c17 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:47.604123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bee25-4ce0-4e69-8c8c-6552f73d0c17 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:47.604303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bee25-4ce0-4e69-8c8c-6552f73d0c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:47.622253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5d6bee25-4ce0-4e69-8c8c-6552f73d0c17 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:47.623420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5d6bee25-4ce0-4e69-8c8c-6552f73d0c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:47.692917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 97/188] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:47.791239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c1eb1-3f19-4067-a52a-cb5202d705e0 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:47.791584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c1eb1-3f19-4067-a52a-cb5202d705e0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:47.791774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c1eb1-3f19-4067-a52a-cb5202d705e0 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:47.793275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 92/189] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:47.798195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:47.798481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:47.798753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:47.817277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:47.818461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:47.890154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 98/190] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:47.959884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c1eb1-3f19-4067-a52a-cb5202d705e0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.056620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c1eb1-3f19-4067-a52a-cb5202d705e0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.090031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.090662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.090940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.091212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.101327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:48.106081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:48.106896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:48.107690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:48.111852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 93/191] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:48.113256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.158047 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.174678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:48.186502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:48.220452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1910-df0e-4704-b6fe-f10644db58da None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.261185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.261803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.262082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.262360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.263022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:48.263326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:48.264738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:48.271263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb4341-30fd-44b7-b5d8-ace08bd3dba8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.273584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 99/192] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:48.279019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.309206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.310648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.311393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.311867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.313217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:48.314874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:48.316643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 94/193] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:48.322125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.388157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3996-6d45-4ef5-af03-e5297278c4f4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.409595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:48.409850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:48.418708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413e5f-44a4-4dcd-95ae-5186923ee9b7 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.421114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b4f9d-b517-4cd5-befb-9dd3d9f0f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.421738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b4f9d-b517-4cd5-befb-9dd3d9f0f41b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.422014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b4f9d-b517-4cd5-befb-9dd3d9f0f41b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.422280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b4f9d-b517-4cd5-befb-9dd3d9f0f41b None admin] PATH_INFO: `/v3/users/58d980b7d56e418a83fd07202de29a99` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.435642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b4f9d-b517-4cd5-befb-9dd3d9f0f41b None admin] RBAC: Authorizing `identity:get_user(user_id=58d980b7d56e418a83fd07202de29a99)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:48.437247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b4f9d-b517-4cd5-befb-9dd3d9f0f41b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:48.444558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 100/194] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:48 2022] GET /identity/v3/users/58d980b7d56e418a83fd07202de29a99 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:48.444558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7705f72-3cfb-4d5b-976a-f4a8e079b3f2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.444558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7705f72-3cfb-4d5b-976a-f4a8e079b3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.444558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7705f72-3cfb-4d5b-976a-f4a8e079b3f2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.444558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 101/195] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:48.446791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7705f72-3cfb-4d5b-976a-f4a8e079b3f2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.456451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09319a65-22db-453c-bd16-541a0017c692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.457068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09319a65-22db-453c-bd16-541a0017c692 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.457345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09319a65-22db-453c-bd16-541a0017c692 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.457617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09319a65-22db-453c-bd16-541a0017c692 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.458273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09319a65-22db-453c-bd16-541a0017c692 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:48.459728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09319a65-22db-453c-bd16-541a0017c692 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:48.544680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7705f72-3cfb-4d5b-976a-f4a8e079b3f2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.548089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 95/196] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:48 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:48.548089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.548089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.548089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.571630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:48.573057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:48.589589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.590541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.590822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.591181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.592402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:48.593865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:48.595721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 102/197] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:48 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:48.602224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:48.648832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 96/198] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:48.657844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:48.712132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e82f0-aefa-416c-80b7-8c623120cd68 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:48.748308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:48.748977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:48.749250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:48.749574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/58d980b7d56e418a83fd07202de29a99/roles/7a8d99c2f3c1479fad1ceeae9a792694` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:48.751412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6745b8eec4c16b05969f452c0ebe6, user_id=58d980b7d56e418a83fd07202de29a99, role_id=7a8d99c2f3c1479fad1ceeae9a792694)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:48.752872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:48.817541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 103/199] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:49:48 2022] PUT /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/users/58d980b7d56e418a83fd07202de29a99/roles/7a8d99c2f3c1479fad1ceeae9a792694 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:48.926811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:49.015660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:49.037657 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eeaa5-0d98-4a9c-a83e-8fc32714f117 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:49.073365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:49.074008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:49.074285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:49.074555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:49.086274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:49.091186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:49.092061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:49.093019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 97/200] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:49.100031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:49.205035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9816e2d-ff09-470e-9294-b5ab00af3207 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:49.215915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:49.241252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:49.241878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:49.242159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:49.242432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:49.243098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:49.243402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:49.244912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:49.258848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 104/201] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:49.260284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:49.364175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e8df-c85e-44eb-b6b5-8abf97f08eee None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:49.399126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:49.399769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:49.400067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:49.400350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] PATH_INFO: `/v3/users/1b54bdb0a2854da6b4f1cd52fcf3ea61` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:49.401522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] RBAC: Authorizing `identity:get_user(user_id=1b54bdb0a2854da6b4f1cd52fcf3ea61)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:49.403134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:49.404948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 98/202] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:49 2022] GET /identity/v3/users/1b54bdb0a2854da6b4f1cd52fcf3ea61 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:49.410556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:49.516799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0cbb3-e472-498a-8781-1dc3a2e813d4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:49.551319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:49.551935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:49.552252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:49.552529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:49.593862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:49.626003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:49.627597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:49.733393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 105/203] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:49:49.739293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:49.786894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:49.879261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2a8c6-c0eb-47fd-8bd3-b36d7d727169 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:49.917006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbed2c-073c-4774-83c7-5bb426c7de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:49.917649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbed2c-073c-4774-83c7-5bb426c7de0a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:49.917929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbed2c-073c-4774-83c7-5bb426c7de0a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:49.918198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbed2c-073c-4774-83c7-5bb426c7de0a None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:49.919327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbed2c-073c-4774-83c7-5bb426c7de0a None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:49.920850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbed2c-073c-4774-83c7-5bb426c7de0a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:49.922652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 99/204] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:49 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:49.928935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:49.998464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:50.029171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b9ea0-95ad-4805-9d5c-3b69b2e93127 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:50.070443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:50.070976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:50.071257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:50.071536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:50.072368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] RBAC: Adding query filter params (role.id=3a658e34b7d14750a0169f576d87baef, scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6, user.id=1b54bdb0a2854da6b4f1cd52fcf3ea61) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:50.072680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:50.074105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170e717-7895-4f11-bc4c-aaea1e7760d5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:50.080360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 106/205] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:49 2022] GET /identity/v3/role_assignments?user.id=1b54bdb0a2854da6b4f1cd52fcf3ea61&scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6&role.id=3a658e34b7d14750a0169f576d87baef => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:50.927226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f2e00-797c-4256-88e0-c078ec15f41d None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:50.927780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f2e00-797c-4256-88e0-c078ec15f41d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:50.927780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f2e00-797c-4256-88e0-c078ec15f41d None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:50.928777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24d6c1-f5dd-4fb3-bfa5-24ff8c0c0d2c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:50.929140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24d6c1-f5dd-4fb3-bfa5-24ff8c0c0d2c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:50.929226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 100/206] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:50.929342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24d6c1-f5dd-4fb3-bfa5-24ff8c0c0d2c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:50.930957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 107/207] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:50.935001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f78b-8f83-4b5d-a0ea-1a6574f10b07 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:50.935280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f78b-8f83-4b5d-a0ea-1a6574f10b07 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:50.935550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f78b-8f83-4b5d-a0ea-1a6574f10b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:50.935656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f500c3-d1b0-46a1-b550-ea5c24117af3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:50.935896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f500c3-d1b0-46a1-b550-ea5c24117af3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:50.936211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f500c3-d1b0-46a1-b550-ea5c24117af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:50.953767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-12f500c3-d1b0-46a1-b550-ea5c24117af3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:50.954208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-cce1f78b-8f83-4b5d-a0ea-1a6574f10b07 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:50.954957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-12f500c3-d1b0-46a1-b550-ea5c24117af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:50.955343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-cce1f78b-8f83-4b5d-a0ea-1a6574f10b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:51.030806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 101/208] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:51.033934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 108/209] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:51.098936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e94013-8a06-4ccd-b30f-06ee39f05a20 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.099251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e94013-8a06-4ccd-b30f-06ee39f05a20 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.099489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e94013-8a06-4ccd-b30f-06ee39f05a20 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.101563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 102/210] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:51.106477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.106767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.107042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.125271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:51.126379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:51.185955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:51.194927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 109/211] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:51.298467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e94013-8a06-4ccd-b30f-06ee39f05a20 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.303284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:51.305709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.410392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d47a11-c6e6-42f3-b596-e1eb0048a7d5 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.414073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e94013-8a06-4ccd-b30f-06ee39f05a20 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.445869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.446521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.446784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.447061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.450286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.451101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.451572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.452019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.453351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:51.454955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:51.456956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 103/212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:51.460636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:51.462704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:51.467137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:51.468218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:51.469017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 110/213] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:51.469681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.526180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:51.533926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:51.562370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158cee-6259-494e-b5b6-d6347bf3741e None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.571815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdca35f-d27f-4d0f-a6e6-7466bb8020d4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.595154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.595765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.596068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.596348 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.597002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:51.598426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:51.604435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.605053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.605327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.605591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.616037 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:51.621207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:51.622105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:51.622903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 111/214] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:51.623553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.690926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 104/215] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:49:51 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:51.691752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.727520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa885e-85e6-47a3-9549-09b2935ff72a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.744163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:51.768092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.768733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.769013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.769372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.770156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] RBAC: Adding query filter params (name=member) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:51.770460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:51.771886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:51.782081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 112/216] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:51.789350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.808082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af724e-2a5f-444b-aa6b-e42373e368a5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.837452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:51.846497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.847123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.847914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.848212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.848917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:51.849212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:51.850658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:51.860843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 105/217] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:51.868257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.911497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55c6a-25aa-46c7-82d6-354b211c5ebd None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:51.959814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:51.960466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:51.960687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:51.961172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] PATH_INFO: `/v3/users/fd556b66515b4c90ae2e8592687a8752` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:51.978361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] RBAC: Authorizing `identity:get_user(user_id=fd556b66515b4c90ae2e8592687a8752)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:51.979971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:51.981840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 113/218] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/users/fd556b66515b4c90ae2e8592687a8752 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:51.989058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:51.994248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08841a-1272-46cd-9074-3851cb97cb6c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:52.034684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:52.039467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:52.040848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:52.040848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:52.040848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] PATH_INFO: `/v3/users/58d980b7d56e418a83fd07202de29a99` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:52.042131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] RBAC: Authorizing `identity:get_user(user_id=58d980b7d56e418a83fd07202de29a99)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:52.043340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:52.048189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 106/219] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/users/58d980b7d56e418a83fd07202de29a99 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:52.051429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:52.122489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a3a9-c278-4f94-96f7-47a129ba7e24 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:52.128150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:52.171248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:52.171488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183964e-1126-48f8-af2d-c6418458dd35 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:52.171921 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:52.172311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:52.172587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:52.173485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] RBAC: Adding query filter params (role.id=3a658e34b7d14750a0169f576d87baef, user.id=fd556b66515b4c90ae2e8592687a8752, scope.system=all) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:52.173796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:52.176068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:52.188251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 114/220] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 10 16:49:51 2022] GET /identity/v3/role_assignments?user.id=fd556b66515b4c90ae2e8592687a8752&scope.system=all&role.id=3a658e34b7d14750a0169f576d87baef => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:52.217879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:52.218432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:52.218712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:52.218987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:52.301362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:52.303420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:52.378953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:52.425014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 107/221] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 16:49:52.430711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:52.536425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:52.541067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879334-ab38-4a38-a454-b8e902a180a9 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:52.577488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8beed3-4508-42ec-b48a-e488087c985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:52.578127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8beed3-4508-42ec-b48a-e488087c985a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:52.578405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8beed3-4508-42ec-b48a-e488087c985a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:52.578678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8beed3-4508-42ec-b48a-e488087c985a None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:52.579809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8beed3-4508-42ec-b48a-e488087c985a None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:52.581285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8beed3-4508-42ec-b48a-e488087c985a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:52.582927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 115/222] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:49:52 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:52.589386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:52.694903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2e9cb-0368-44e7-b3d6-bb60f543f427 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:52.735426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:52.736553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:52.737038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:52.737537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:52.738525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] RBAC: Adding query filter params (role.id=7a8d99c2f3c1479fad1ceeae9a792694, scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6, user.id=58d980b7d56e418a83fd07202de29a99) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:52.739034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:52.740741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaddb16-ef95-48fa-924a-e3604ca3f04e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:52.746913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 108/223] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:49:52 2022] GET /identity/v3/role_assignments?user.id=58d980b7d56e418a83fd07202de29a99&scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6&role.id=7a8d99c2f3c1479fad1ceeae9a792694 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:53.963299 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec93448-ff52-4db1-b355-05ce61c51697 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:53.963842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec93448-ff52-4db1-b355-05ce61c51697 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:53.963842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec93448-ff52-4db1-b355-05ce61c51697 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:53.965251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 116/224] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:53.971264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b5ab-20a4-4c74-8ae4-f9d4315f8005 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:53.971724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b5ab-20a4-4c74-8ae4-f9d4315f8005 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:53.971794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b5ab-20a4-4c74-8ae4-f9d4315f8005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:53.989463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e362b5ab-20a4-4c74-8ae4-f9d4315f8005 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:53.991171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e362b5ab-20a4-4c74-8ae4-f9d4315f8005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:54.008471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:54.059045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 109/225] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:54.059709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514162c-7c14-48bc-ba52-c3223fd70fea None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.059996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514162c-7c14-48bc-ba52-c3223fd70fea None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.060305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514162c-7c14-48bc-ba52-c3223fd70fea None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.061755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 117/226] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:49:54.066667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef2385-37d6-4dbb-ac68-df67bdca033d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.066959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef2385-37d6-4dbb-ac68-df67bdca033d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.067233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef2385-37d6-4dbb-ac68-df67bdca033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.085590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bfef2385-37d6-4dbb-ac68-df67bdca033d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:54.086776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bfef2385-37d6-4dbb-ac68-df67bdca033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:54.153172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 110/227] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:54.323030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514162c-7c14-48bc-ba52-c3223fd70fea None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.398239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faeadd7-1ab5-4e18-b890-c0340651f797 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.398663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faeadd7-1ab5-4e18-b890-c0340651f797 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.399114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faeadd7-1ab5-4e18-b890-c0340651f797 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.400527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 111/228] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:54.405582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.405816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.406097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.424184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:54.425420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:54.425912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514162c-7c14-48bc-ba52-c3223fd70fea None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.459713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.460365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.460644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.460907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.470506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:54.475752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:54.476661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:49:54.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 118/229] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:54.478569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.495653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 112/230] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:54.496724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.510098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:54.511896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:54.585213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e2186-4f66-45a0-babf-0fa02732c7b9 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.596277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27262b1d-99c7-48f5-a7b8-c24566cfd01c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.618847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.619476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.619754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.620052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.621151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:54.622583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:54.624363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 119/231] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:54.629546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.635318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.635926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.636230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.636505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.637162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:54.637461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:54.638852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:54.647438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 113/232] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:54.652915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.722186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:54.729513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:54.733244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9d0ed-2174-4159-bb0b-46ed00cfa8c0 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.755314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06c250-ee5c-45a7-96cf-448c91c726bd None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.770090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.770740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.771027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.771300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] PATH_INFO: `/v3/groups` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.772491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:54.773917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:54.789795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.790421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.794195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.794195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] PATH_INFO: `/v3/users/6d7f94b58d984e189da5f8fd12f595f4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.803702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] RBAC: Authorizing `identity:get_user(user_id=6d7f94b58d984e189da5f8fd12f595f4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:54.805267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:54.807102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 114/233] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/users/6d7f94b58d984e189da5f8fd12f595f4 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:54.807933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.827952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 120/234] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Oct 10 16:49:54 2022] POST /identity/v3/groups => generated 229 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:54.833094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:54.925420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14d05-6e84-4356-8706-896e2d7defe1 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.955777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e69a9-7165-4afc-b343-d406b9d398f3 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:54.961947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:54.969125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.974703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.974703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.974703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.985425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:54.991511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:54.992447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:54.993899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 115/235] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:54.994580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:54.994580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:54.995057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:54.995057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:54.995790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] RBAC: Adding query filter params (role.id=7a8d99c2f3c1479fad1ceeae9a792694, user.id=6d7f94b58d984e189da5f8fd12f595f4, scope.system=all) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:54.996134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:54.997590 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:55.000576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:55.003036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 121/236] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/role_assignments?user.id=6d7f94b58d984e189da5f8fd12f595f4&scope.system=all&role.id=7a8d99c2f3c1479fad1ceeae9a792694 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:55.114967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602709-995d-4fe9-8117-3106a4a74b7e None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:55.151538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:55.152361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:55.152744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:55.153119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:55.153903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:55.154310 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:55.155807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:55.165117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 116/237] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:55.170904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:55.191427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:55.276245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc895e-2140-4f22-bc3e-d4399ebb96bf None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:55.309226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1305e-1c7d-40a4-af9c-5cc90ac07d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:55.309858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1305e-1c7d-40a4-af9c-5cc90ac07d7a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:55.310138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1305e-1c7d-40a4-af9c-5cc90ac07d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:55.310414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1305e-1c7d-40a4-af9c-5cc90ac07d7a None admin] PATH_INFO: `/v3/users/fd556b66515b4c90ae2e8592687a8752` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:55.311525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1305e-1c7d-40a4-af9c-5cc90ac07d7a None admin] RBAC: Authorizing `identity:get_user(user_id=fd556b66515b4c90ae2e8592687a8752)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:55.312993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1305e-1c7d-40a4-af9c-5cc90ac07d7a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:55.314860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 122/238] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:55 2022] GET /identity/v3/users/fd556b66515b4c90ae2e8592687a8752 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:55.321045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:55.380384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:55.423465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885ae36-0abf-44b7-8ae1-e6aaee203303 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:55.456962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7f0b1-744f-40fa-a744-a3c9fb36ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:55.457610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7f0b1-744f-40fa-a744-a3c9fb36ee73 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:55.457884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7f0b1-744f-40fa-a744-a3c9fb36ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:55.458155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7f0b1-744f-40fa-a744-a3c9fb36ee73 None admin] PATH_INFO: `/v3/system/users/fd556b66515b4c90ae2e8592687a8752/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:55.459556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7f0b1-744f-40fa-a744-a3c9fb36ee73 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fd556b66515b4c90ae2e8592687a8752, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:55.460962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7f0b1-744f-40fa-a744-a3c9fb36ee73 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:55.558674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 117/239] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Oct 10 16:49:55 2022] PUT /identity/v3/system/users/fd556b66515b4c90ae2e8592687a8752/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:57.044769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c28e-2a35-44e8-8dcf-bc86e46b3047 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.045301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c28e-2a35-44e8-8dcf-bc86e46b3047 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.045301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c28e-2a35-44e8-8dcf-bc86e46b3047 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.046936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 123/240] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:57.053120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecfd6-da55-445a-9c7c-576e7dcf7262 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.053622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecfd6-da55-445a-9c7c-576e7dcf7262 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.053692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecfd6-da55-445a-9c7c-576e7dcf7262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.074099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3edecfd6-da55-445a-9c7c-576e7dcf7262 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:57.075330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3edecfd6-da55-445a-9c7c-576e7dcf7262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:57.143012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 118/241] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:57.171132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8349277-47eb-421f-baeb-687713861c07 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.171419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8349277-47eb-421f-baeb-687713861c07 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.171702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8349277-47eb-421f-baeb-687713861c07 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.173228 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 124/242] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:57.178702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.178982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.179247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.198031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:57.199201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:57.243043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:57.268078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 119/243] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:49:57.415140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8349277-47eb-421f-baeb-687713861c07 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:57.502905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:57.518874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8349277-47eb-421f-baeb-687713861c07 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:57.533432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:57.560829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:57.561495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.561778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.562049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.563238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:57.564766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:57.568082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 125/244] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:49:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:57.572596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:57.644204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec051-ba89-4764-9f6b-62c5aa908fbe None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:57.677755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f920b-3884-4c0d-bd5d-c103522a43af None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:57.679850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62254e10-1aae-4e5f-b851-74aaaa5777aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:57.680935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62254e10-1aae-4e5f-b851-74aaaa5777aa None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.681203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62254e10-1aae-4e5f-b851-74aaaa5777aa None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.681469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62254e10-1aae-4e5f-b851-74aaaa5777aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.691359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62254e10-1aae-4e5f-b851-74aaaa5777aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-62254e10-1aae-4e5f-b851-74aaaa5777aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:57.696250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:57.697218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:57.697991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 120/245] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:49:57.698947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d2c16-1171-45be-a269-1b3ce7698cfb None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.699677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d2c16-1171-45be-a269-1b3ce7698cfb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.699964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d2c16-1171-45be-a269-1b3ce7698cfb None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.701403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 121/246] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:49:57.703535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d2c16-1171-45be-a269-1b3ce7698cfb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:57.713716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47add9a3-d39c-4a81-9f07-eced860ae709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:57.714325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47add9a3-d39c-4a81-9f07-eced860ae709 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.714597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47add9a3-d39c-4a81-9f07-eced860ae709 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.714865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47add9a3-d39c-4a81-9f07-eced860ae709 None admin] PATH_INFO: `/v3/groups` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.715480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47add9a3-d39c-4a81-9f07-eced860ae709 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:57.717003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47add9a3-d39c-4a81-9f07-eced860ae709 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:57.743867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:57.794091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 126/247] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Oct 10 16:49:57 2022] POST /identity/v3/groups => generated 226 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:49:57.794091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67216c58-2b05-43dc-9497-45af745760db None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.794091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67216c58-2b05-43dc-9497-45af745760db None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.794091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67216c58-2b05-43dc-9497-45af745760db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.800981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d2c16-1171-45be-a269-1b3ce7698cfb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:57.815605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-67216c58-2b05-43dc-9497-45af745760db None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:49:57.817030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-67216c58-2b05-43dc-9497-45af745760db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:49:57.827840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:57.850255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:57.851164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:57.851445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:57.851720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:57.852593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:57.852832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:57.854338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:57.873694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 122/248] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:49:57.879475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:57.890682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 127/249] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:49:57.991880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:57.995935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae9e45-34da-46cc-924d-647fd39bcefd None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:58.032979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:58.033617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:58.033899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:58.034167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] PATH_INFO: `/v3/users/6d7f94b58d984e189da5f8fd12f595f4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:58.035307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] RBAC: Authorizing `identity:get_user(user_id=6d7f94b58d984e189da5f8fd12f595f4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:58.036807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:58.038607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 123/250] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:57 2022] GET /identity/v3/users/6d7f94b58d984e189da5f8fd12f595f4 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:58.044124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67216c58-2b05-43dc-9497-45af745760db None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:58.151562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67216c58-2b05-43dc-9497-45af745760db None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:58.160528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:58.196755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:58.197419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:58.197699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:58.198238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] PATH_INFO: `/v3/system/users/6d7f94b58d984e189da5f8fd12f595f4/roles/7a8d99c2f3c1479fad1ceeae9a792694` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:58.199379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6d7f94b58d984e189da5f8fd12f595f4, role_id=7a8d99c2f3c1479fad1ceeae9a792694)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:58.200813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:58.267113 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 128/251] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Oct 10 16:49:58 2022] PUT /identity/v3/system/users/6d7f94b58d984e189da5f8fd12f595f4/roles/7a8d99c2f3c1479fad1ceeae9a792694 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:49:58.285135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b1ab5-db64-4901-a88b-c692aa30cde0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:58.333752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:58.334439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:58.334720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:58.334989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:58.347511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:49:58.352761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:49:58.353668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:49:58.354442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 124/252] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:49:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:49:58.360013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:58.373647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:58.479417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1445c1-a710-4b21-bb1c-e1b529e7f683 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:58.517838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:58.518372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:58.518645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:58.518964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:58.519680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] RBAC: Adding query filter params (name=member) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:58.519982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:58.522084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:58.530552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 129/253] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:49:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:58.536087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:58.575956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:58.663730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9c7ce-e998-487f-ac2c-cf2b9f272324 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:58.705323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689ea28-7f9b-416c-af28-a06bba2413f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:58.706256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689ea28-7f9b-416c-af28-a06bba2413f0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:58.706693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689ea28-7f9b-416c-af28-a06bba2413f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:58.707122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689ea28-7f9b-416c-af28-a06bba2413f0 None admin] PATH_INFO: `/v3/users/fd556b66515b4c90ae2e8592687a8752` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:58.710692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689ea28-7f9b-416c-af28-a06bba2413f0 None admin] RBAC: Authorizing `identity:get_user(user_id=fd556b66515b4c90ae2e8592687a8752)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:58.712364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689ea28-7f9b-416c-af28-a06bba2413f0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:58.714179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 125/254] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:49:58 2022] GET /identity/v3/users/fd556b66515b4c90ae2e8592687a8752 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:49:58.720763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:49:58.797482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:49:58.838404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac0aa-6c12-4a1f-a878-037aa295d69f None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:49:58.876266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:49:58.876896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:49:58.877173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:49:58.877444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:49:58.878179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] RBAC: Adding query filter params (role.id=3a658e34b7d14750a0169f576d87baef, user.id=fd556b66515b4c90ae2e8592687a8752, scope.system=all) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:49:58.878490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:49:58.879935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f72e-5b14-4b14-ad4c-c07db2e0a472 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:49:58.893072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 130/255] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 10 16:49:58 2022] GET /identity/v3/role_assignments?user.id=fd556b66515b4c90ae2e8592687a8752&scope.system=all&role.id=3a658e34b7d14750a0169f576d87baef => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:00.334792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803c3da-3cf1-4478-8073-9d01b527893e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:00.335307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803c3da-3cf1-4478-8073-9d01b527893e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:00.335381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803c3da-3cf1-4478-8073-9d01b527893e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:00.336791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 126/256] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:00.342865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471949d-cddd-44c5-a69f-1e2c4d43cbdc None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:00.343320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471949d-cddd-44c5-a69f-1e2c4d43cbdc None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:00.343409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471949d-cddd-44c5-a69f-1e2c4d43cbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:00.368245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2471949d-cddd-44c5-a69f-1e2c4d43cbdc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:00.369449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2471949d-cddd-44c5-a69f-1e2c4d43cbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:00.454742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 131/257] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 16:50:00.518651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d261bde-b2b5-4cc1-9285-513ebc88683e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:00.518947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d261bde-b2b5-4cc1-9285-513ebc88683e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:00.519212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d261bde-b2b5-4cc1-9285-513ebc88683e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:00.520815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 127/258] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:00.527393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:00.527683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:00.527955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:00.557028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:00.558225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:00.566224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 132/259] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 16:50:00.995890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d261bde-b2b5-4cc1-9285-513ebc88683e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.012547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d261bde-b2b5-4cc1-9285-513ebc88683e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.017335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.018205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.018703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.019165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.042233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:01.047378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:01.048281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:01.049038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:01.051287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 128/260] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:50:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:01.057480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.074510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78d5db-fa5d-447f-a246-6ccc7b10f45f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.083651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.084353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.084634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.084906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.085580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:01.085882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.087688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 16:50:01.087688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 16:50:01.087688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 16:50:01.088638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.096406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.107922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 133/261] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:01.115095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaeca8-18c9-465e-9f00-45ef1fcde834 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.119557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.120220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.120499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.120771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.126663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.139488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec576e3c-f27e-482e-8d93-4a61bfa1fce9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.140715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:01.149209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:01.150103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:01.150868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 129/262] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:01.151508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.152112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.152112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.155899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.163360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.164428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 16:50:01.164428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 16:50:01.164428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 16:50:01.165649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.167638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 134/263] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:01.173408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.179985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.189869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7e866-704b-40fc-96f3-c42461fc8130 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.196269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.196903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.197178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.197448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.198862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.200972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.202635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 130/264] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:01.211081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afaf3-1eee-4566-9a66-8d0d26e93508 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.214574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.218532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.219203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.219480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.219753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.220500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:01.220809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.227653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.242661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785c746-6286-451e-850b-1b17b0116305 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.252057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.252384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 135/265] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:01.253014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.253357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.253630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/groups/1e3e6da9920843f293f9b07beea59a05/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.255555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d99d3c78f8497084bf8a42ad9bb84e, group_id=1e3e6da9920843f293f9b07beea59a05, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.259698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.260516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 16:50:01.260516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 16:50:01.260516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 16:50:01.262540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690c070-7244-45eb-9f32-621e46bd9b52 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.276010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58a38d-5aca-48cd-b711-ce7a922f367e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.281979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.294758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.294758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.294758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.294758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.294758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.294758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 136/266] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:01.295785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.306284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8d5ea-616d-42f9-a62f-ec39918ffde5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.312099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.313286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.313286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.313632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.314461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.316560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.318428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 137/267] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:01.324197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.341499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca74bd-d239-44f1-add5-e3cf9d0014fd admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:01.346095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:01.346771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.347052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.347332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/groups/1e3e6da9920843f293f9b07beea59a05/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.349569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d99d3c78f8497084bf8a42ad9bb84e, group_id=1e3e6da9920843f293f9b07beea59a05, role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:01.350599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 16:50:01.350599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 16:50:01.350599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 16:50:01.352674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ffdd4c-61da-4e71-9db8-c53cbae4c6ca admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:01.386567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 131/268] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 10 16:50:01 2022] PUT /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/groups/1e3e6da9920843f293f9b07beea59a05/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:01.386567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1161150-b043-4d02-bec4-3b3a01b93875 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.386567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1161150-b043-4d02-bec4-3b3a01b93875 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.386567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1161150-b043-4d02-bec4-3b3a01b93875 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.387628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 132/269] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:01.394170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.394514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.394720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.420150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:01.421450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:01.451483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 138/270] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 10 16:50:01 2022] PUT /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e/groups/1e3e6da9920843f293f9b07beea59a05/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:01.453093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970fa37-e04b-4ddc-84a9-07568643c96c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.453396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970fa37-e04b-4ddc-84a9-07568643c96c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.453673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970fa37-e04b-4ddc-84a9-07568643c96c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.455058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 139/271] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:01.466281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:01.466617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:01.466833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:01.497624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:01.499025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:01.526390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:01.526822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 133/272] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 10 16:50:01.601629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 140/273] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:50:01.938098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:01.940822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:02.061961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fee68-7710-471a-8dcb-886e3206f651 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.099808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.100471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.100746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.101014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.115491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:02.120262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:02.121131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:02.121913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 134/274] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:02.127501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.244548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95bf3c-7bbb-49c0-aac4-1f93f981e2fc None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.265662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.300438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.301061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.301337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.301606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.302267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:02.303101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.304555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.319709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 141/275] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:02.325577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.347438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:02.388780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680733-43a0-44fc-87b6-1da0a1237010 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.393238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.393887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.394168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.394442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.416254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:02.421301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:02.422163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:02.422939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 135/276] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:02.429623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.437972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:02.450794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7155c8-ce20-4df1-9a67-c5047abc8257 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.459962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.460651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.461136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.463430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784df9e-bc21-48df-9716-ec90946691c6 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.464183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.464956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:02.465270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.466366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 16:50:02.466366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 16:50:02.466366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 16:50:02.467448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.483842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 136/277] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:02.489331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.512079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c66fe-ba10-4ef6-a87e-0249ddfd4929 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.512297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.513036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.513318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.513588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.515192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.516416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 16:50:02.516416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 16:50:02.516416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 16:50:02.517318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.517692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.518295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.518707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.519123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] PATH_INFO: `/v3/users/6d7f94b58d984e189da5f8fd12f595f4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.519337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 137/278] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:02.524787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] RBAC: Authorizing `identity:get_user(user_id=6d7f94b58d984e189da5f8fd12f595f4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.524988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.526571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.528672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 142/279] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/users/6d7f94b58d984e189da5f8fd12f595f4 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:02.537475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.540863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f52f8-4957-4bed-ad48-a191be4d05e3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.545610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.546394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.546674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.546947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.548116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.550309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.557199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 138/280] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:02.562131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:02.579474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4011-10d3-4df4-8f4a-19bb5e41d970 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.584457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad607e-d505-4a44-965a-fa6fe90bf359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.585276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad607e-d505-4a44-965a-fa6fe90bf359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.585531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad607e-d505-4a44-965a-fa6fe90bf359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.585808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad607e-d505-4a44-965a-fa6fe90bf359 admin admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/groups/1e3e6da9920843f293f9b07beea59a05/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.587796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad607e-d505-4a44-965a-fa6fe90bf359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6745b8eec4c16b05969f452c0ebe6, group_id=1e3e6da9920843f293f9b07beea59a05, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.594904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad607e-d505-4a44-965a-fa6fe90bf359 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.684016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542651fe-d423-4f2e-80e7-fe7b89df09fa None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:02.687448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:02.690337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 139/281] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 10 16:50:02 2022] PUT /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/groups/1e3e6da9920843f293f9b07beea59a05/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:50:02.738425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:02.739056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.739332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.739652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.741033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] RBAC: Adding query filter params (role.id=7a8d99c2f3c1479fad1ceeae9a792694, user.id=6d7f94b58d984e189da5f8fd12f595f4, scope.system=all) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:02.741339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:02.742767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d458de-9156-4c34-a735-929122f94e9f None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:02.759694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 143/282] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 10 16:50:02 2022] GET /identity/v3/role_assignments?user.id=6d7f94b58d984e189da5f8fd12f595f4&scope.system=all&role.id=7a8d99c2f3c1479fad1ceeae9a792694 => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:02.810264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045c0a-effe-44e2-8714-6dc13cbaee35 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.810589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045c0a-effe-44e2-8714-6dc13cbaee35 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.810871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045c0a-effe-44e2-8714-6dc13cbaee35 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.812369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 140/283] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:02.834876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eece0f-fb77-4880-a102-7276140c2925 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:02.835206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eece0f-fb77-4880-a102-7276140c2925 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:02.835432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eece0f-fb77-4880-a102-7276140c2925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:02.880881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-82eece0f-fb77-4880-a102-7276140c2925 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:02.882117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-82eece0f-fb77-4880-a102-7276140c2925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:02.967027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 144/284] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 16:50:03.112262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a18ce-558a-4baf-b8e0-f759ba72da95 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.112595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a18ce-558a-4baf-b8e0-f759ba72da95 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.112894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a18ce-558a-4baf-b8e0-f759ba72da95 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.114245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 141/285] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:03.120359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.120650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.120929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.144485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:03.145677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:03.157965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 145/286] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 16:50:03.265121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a18ce-558a-4baf-b8e0-f759ba72da95 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.278310 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a18ce-558a-4baf-b8e0-f759ba72da95 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.282696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.288785 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.289074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.289408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.300578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:03.314065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:03.316127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:03.318010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 142/287] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:03.324047 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.335147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549395-87a6-42c0-a0a4-a431ccf029d1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.339675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.340321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.340598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.340866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.341527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:03.341827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.343469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.359666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 146/288] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:03.365217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.375256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc574d3c-91b6-4009-be64-731f8dc073f2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.379642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.384353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.384645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.384928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.386083 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.387986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.389578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 143/289] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:03.395780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.409140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d647-be00-4ef0-8bf5-6265831e7769 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.413641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.414280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.414613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.414891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.416017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.418075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.419999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 147/290] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:03.425416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.438522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52781e-eb72-4812-9bf0-31750eb02ce4 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.442953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.444176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.444460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.444793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/groups/1e3e6da9920843f293f9b07beea59a05/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.447042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6745b8eec4c16b05969f452c0ebe6, group_id=1e3e6da9920843f293f9b07beea59a05, role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.449701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.548799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 144/291] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 10 16:50:03 2022] PUT /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6/groups/1e3e6da9920843f293f9b07beea59a05/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:03.647783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.659405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff3d8-0dee-4c7b-a861-492e93f07ead admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.737454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.738205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.738491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.738767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.753615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:03.762661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:03.764829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:03.766807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 148/292] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 117 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:03.770863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.780475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f05ccc-1480-4685-8404-7b92504aef1b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.785024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.785630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.785910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.786185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.786876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:03.787233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.791852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.805926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 145/293] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:03.811577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.820170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:03.824806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf43d-f395-4d18-bd3f-aca01aeb9195 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.829384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.830022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.830303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.835370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] PATH_INFO: `/v3/groups/d7a1003e548b424b958a8b977367ca4a` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.838710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] RBAC: Authorizing `identity:get_group(group_id=d7a1003e548b424b958a8b977367ca4a)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.840669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.842622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 149/294] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/groups/d7a1003e548b424b958a8b977367ca4a => generated 229 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:03.848252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.859763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f24c0-d05b-4fd1-8804-8484bc977156 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.864253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca5bed-3d4a-4610-8a83-fb4b4e1c68c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.865436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca5bed-3d4a-4610-8a83-fb4b4e1c68c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.865731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca5bed-3d4a-4610-8a83-fb4b4e1c68c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.866005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca5bed-3d4a-4610-8a83-fb4b4e1c68c8 admin admin] PATH_INFO: `/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.867192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca5bed-3d4a-4610-8a83-fb4b4e1c68c8 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3acc8f5bcdf4b42bd810be11a69d5c4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.869064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca5bed-3d4a-4610-8a83-fb4b4e1c68c8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.871005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 146/295] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 10 16:50:03 2022] GET /identity/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4 => generated 344 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:03.876628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:03.890537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2cb34-1aee-432c-9f00-8f032fc015da admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:03.895091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f267d-f190-4036-a3a7-c4db879cd98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:03.895777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f267d-f190-4036-a3a7-c4db879cd98c admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:03.896080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f267d-f190-4036-a3a7-c4db879cd98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:03.896407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f267d-f190-4036-a3a7-c4db879cd98c admin admin] PATH_INFO: `/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4/groups/d7a1003e548b424b958a8b977367ca4a/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:03.898325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f267d-f190-4036-a3a7-c4db879cd98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3acc8f5bcdf4b42bd810be11a69d5c4, group_id=d7a1003e548b424b958a8b977367ca4a, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:03.901022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f267d-f190-4036-a3a7-c4db879cd98c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:03.983462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 150/296] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 10 16:50:03 2022] PUT /identity/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4/groups/d7a1003e548b424b958a8b977367ca4a/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:04.666213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8be1d-e315-4244-8465-70e187e400d5 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:04.666909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8be1d-e315-4244-8465-70e187e400d5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:04.666909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8be1d-e315-4244-8465-70e187e400d5 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:04.668291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 147/297] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:04.673643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:04.674131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:04.674282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:04.697681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:04.698863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:04.754657 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e65c56-33e0-4ef1-8449-bb86b702b437 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:04.754974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e65c56-33e0-4ef1-8449-bb86b702b437 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:04.755196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e65c56-33e0-4ef1-8449-bb86b702b437 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:04.757144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 148/298] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:04.769458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:04.769765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:04.770044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:04.801083 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:04.802287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:04.808537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 151/299] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:04.899495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 149/300] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:50:05.235374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:05.356150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:05.395014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881680-b5ee-4bd0-b7ea-c43bd19f0096 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:05.424128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:05.462384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:05.463426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:05.463901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:05.464407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:05.466762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:05.485604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:05.491511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:05.498010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:05.498792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 152/301] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:05.506786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:05.546945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f906d-50e2-42b4-be49-be2fb8f2c659 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:05.631596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:05.632283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:05.632560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:05.632831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:05.644473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b02781-2dc1-40b4-a51f-f1fb46e772d2 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:05.659284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:05.664753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:05.665790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:05.667271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 150/302] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:05.673313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:05.697632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:05.698920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:05.699618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:05.700754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:05.701651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] RBAC: Adding query filter params (name=member) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:05.703072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:05.709049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:05.724085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 153/303] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:50:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:05.732241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:05.801749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:05.881822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31d5e-8479-4a54-8b67-f724b4c831a6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:05.898128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440bab2-a9fe-447a-b242-ff410959377c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:05.909987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:05.932696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:05.933316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:05.933595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:05.933934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:05.934608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:05.935040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:05.936843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fbaef-3a4c-4cfb-b1fa-9132e96ae7e2 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:05.946444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cf5ac-24af-4510-ba18-225372e1c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:05.946958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 151/304] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:05.947592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cf5ac-24af-4510-ba18-225372e1c915 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:05.948174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cf5ac-24af-4510-ba18-225372e1c915 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:05.948277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956470c-6999-447a-be1e-65cbc4966173 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:05.948549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956470c-6999-447a-be1e-65cbc4966173 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:05.948821 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956470c-6999-447a-be1e-65cbc4966173 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:05.949095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cf5ac-24af-4510-ba18-225372e1c915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:05.950804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 152/305] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:05.956643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956470c-6999-447a-be1e-65cbc4966173 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.039110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cf5ac-24af-4510-ba18-225372e1c915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:06.041616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cf5ac-24af-4510-ba18-225372e1c915 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:06.080824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956470c-6999-447a-be1e-65cbc4966173 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:06.086366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:06.135709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:06.136722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.137225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.137690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.172801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 154/306] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:06.177306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.178550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.180779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.209283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:06.209612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:06.211232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:06.232608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:06.234518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:06.304050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 155/307] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:06.304050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.395819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 153/308] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:06.404928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.440618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:06.508835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487405-3e37-4f50-861d-72f0f2ebc721 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:06.565097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:06.576209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cd76f-e723-4df2-97ff-a1d0c641dabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:06.576857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd76f-e723-4df2-97ff-a1d0c641dabb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.577134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd76f-e723-4df2-97ff-a1d0c641dabb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.577408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd76f-e723-4df2-97ff-a1d0c641dabb None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.579136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cd76f-e723-4df2-97ff-a1d0c641dabb None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:06.579970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cd76f-e723-4df2-97ff-a1d0c641dabb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:06.587214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 156/309] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:50:06 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:06.588425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63e652-1693-466b-aa09-477e59b10e0f None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.588425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63e652-1693-466b-aa09-477e59b10e0f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.589190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63e652-1693-466b-aa09-477e59b10e0f None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.589278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afc47-e0d4-4a83-b73d-83b4f775be42 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:06.591046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 157/310] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:06.595845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63e652-1693-466b-aa09-477e59b10e0f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.628137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28d27d-1d6e-418c-ae85-29b6fb28124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:06.628765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28d27d-1d6e-418c-ae85-29b6fb28124f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.629043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28d27d-1d6e-418c-ae85-29b6fb28124f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.632054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28d27d-1d6e-418c-ae85-29b6fb28124f None admin] PATH_INFO: `/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.632054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28d27d-1d6e-418c-ae85-29b6fb28124f None admin] RBAC: Authorizing `identity:get_project(project_id=72d99d3c78f8497084bf8a42ad9bb84e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:06.632054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28d27d-1d6e-418c-ae85-29b6fb28124f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:06.633707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 154/311] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:50:06 2022] GET /identity/v3/projects/72d99d3c78f8497084bf8a42ad9bb84e => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:06.634972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f90819-f9b9-45b2-a837-cde44fc2d5b1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.635257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f90819-f9b9-45b2-a837-cde44fc2d5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.635526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f90819-f9b9-45b2-a837-cde44fc2d5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.654784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-94f90819-f9b9-45b2-a837-cde44fc2d5b1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:06.655907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-94f90819-f9b9-45b2-a837-cde44fc2d5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:06.714164 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63e652-1693-466b-aa09-477e59b10e0f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:06.743964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 155/312] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:06.745107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d40f43-5f12-448a-bd87-9c6c1c44c9fa None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.750196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d40f43-5f12-448a-bd87-9c6c1c44c9fa None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.750196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d40f43-5f12-448a-bd87-9c6c1c44c9fa None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.750196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 156/313] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:06.750196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d40f43-5f12-448a-bd87-9c6c1c44c9fa None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.763484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:06.764682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.764970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.765239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.766335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:06.767829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:06.771873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 158/314] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 10 16:50:06 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:06.771873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.774701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:06.854497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:06.887989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d40f43-5f12-448a-bd87-9c6c1c44c9fa None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:06.913519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25101e-8608-4948-89e8-d6fc0947ffe6 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:06.933485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7158f-41e3-4c96-af98-2dae2267b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:06.934416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7158f-41e3-4c96-af98-2dae2267b941 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.934702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7158f-41e3-4c96-af98-2dae2267b941 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.934969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7158f-41e3-4c96-af98-2dae2267b941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.947842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7158f-41e3-4c96-af98-2dae2267b941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5ab7158f-41e3-4c96-af98-2dae2267b941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:06.952790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:06.953684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:06.954475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 157/315] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:06.955480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.955480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.955714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.963184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:06.963976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:06.964545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:06.965057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:06.966949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:06.968849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:06.971571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 159/316] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 10 16:50:06 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:06.976934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:06.978494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:06.979686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:07.059221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.059406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 158/317] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:07.060801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.099833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da85efc-fe30-40be-a400-acc7e2fcc024 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.142537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.143142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.143422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.143691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.144446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] RBAC: Adding query filter params (group.id=1e3e6da9920843f293f9b07beea59a05, role.id=3a658e34b7d14750a0169f576d87baef, scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:07.144803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:07.146794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:07.152571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 160/318] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 10 16:50:06 2022] GET /identity/v3/role_assignments?group.id=1e3e6da9920843f293f9b07beea59a05&scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e&role.id=3a658e34b7d14750a0169f576d87baef => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:07.153359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.159614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.208148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d20624-b2cf-4f81-87ca-d5013925d55c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.254863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.256436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.257429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.257908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.259283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] RBAC: Adding query filter params (name=member) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:07.260153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:07.262193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:07.275149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 159/319] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:07.279889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.295455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d630be-6345-4530-bedc-2da261d8c2f7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.342870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.343748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.344146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.344518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.345486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] RBAC: Adding query filter params (group.id=1e3e6da9920843f293f9b07beea59a05, role.id=215077d32ef84d5eaa4d487616aa2ec3, scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:07.345898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:07.347591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:07.349767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.355073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 161/320] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/role_assignments?group.id=1e3e6da9920843f293f9b07beea59a05&scope.project.id=72d99d3c78f8497084bf8a42ad9bb84e&role.id=215077d32ef84d5eaa4d487616aa2ec3 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:07.357103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.408491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c88a4-0eaf-4acc-8667-8780b7c4c984 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.444313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.450059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.450944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.451392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.451803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.465543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:07.469535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:07.470451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:07.471212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 160/321] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:07.471926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.475601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265c200-fad4-41a8-8236-ac560a58938e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.517316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.517944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.518223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.518499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.600876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79af835-6ebe-4b7a-8fa9-6f1765cfb8d8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.604856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.608926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:07.610383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:07.652127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.652764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.653044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.653315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.665476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:07.670774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:07.671625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:07.672499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 161/322] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:07.673149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.732171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.733994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 162/323] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:07.736652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.789874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc2ce6-9a8a-4142-abe2-d97a04fe8747 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.827375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.827992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.828407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.828574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.829242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:07.829547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:07.830910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:07.844067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 162/324] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:07.844067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.844290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.847445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2058b-623c-41cd-a9c2-34a9bc67e321 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.897632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.898237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.898513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.898788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.899449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:07.899749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:07.901214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:07.910826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 163/325] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:07.912694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:07.956742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e3b48-83e5-4165-8d4c-cf8c4cecf72c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:07.974841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:07.999333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:07.999333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:07.999333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:07.999333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:07.999697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.000823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.002649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 163/326] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.007376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.044537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874caf6c-c2fc-4dfc-8fa7-823e41734221 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.082645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.083262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.084132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.084362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.111418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.117522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa9739-3319-482c-88e5-0a4f61febb45 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.150013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.150417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.150611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.150891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.151161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.151853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.198256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.216159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.217634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.253623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 164/327] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:08.254426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.311042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.313628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 164/328] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:08.314461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.350615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156cfd-01a5-474a-a810-4742866c2036 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.382846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.383469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.383746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.384016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.385128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.386543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.388122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 165/329] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.388877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.399661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.417476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6f25b-0838-4d8e-8018-4de3fd8f1903 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.451214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.451841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.452149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.452421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] PATH_INFO: `/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.453532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] RBAC: Authorizing `identity:get_project(project_id=b3a6745b8eec4c16b05969f452c0ebe6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.454947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.456604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 165/330] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/projects/b3a6745b8eec4c16b05969f452c0ebe6 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.457365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.487917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545700-a88f-4457-a009-38f2f90ab89e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.513328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.520767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.526392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.526392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.526392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] PATH_INFO: `/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.526392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] RBAC: Authorizing `identity:get_project(project_id=b3acc8f5bcdf4b42bd810be11a69d5c4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.526392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.527061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 166/331] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/projects/b3acc8f5bcdf4b42bd810be11a69d5c4 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.532375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.555313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a584a3-aff1-4ff7-a104-66332b13758a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.589134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.589831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.590179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.590526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.591318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] RBAC: Adding query filter params (group.id=1e3e6da9920843f293f9b07beea59a05, role.id=3a658e34b7d14750a0169f576d87baef, scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:08.591619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.593085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.599341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 166/332] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/role_assignments?group.id=1e3e6da9920843f293f9b07beea59a05&scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6&role.id=3a658e34b7d14750a0169f576d87baef => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.600168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.617267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.646856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c26f8-dbc4-43c3-8d97-9f4a3117cd4d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.688079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.688757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.689036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.689303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] PATH_INFO: `/v3/groups/1e3e6da9920843f293f9b07beea59a05` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.690476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] RBAC: Authorizing `identity:get_group(group_id=1e3e6da9920843f293f9b07beea59a05)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.691941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.694373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 167/333] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/groups/1e3e6da9920843f293f9b07beea59a05 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.700195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.712977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2718b7-8476-46a9-8c7e-869ab97c91c5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.754255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.754906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.755181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.755986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] PATH_INFO: `/v3/groups/d7a1003e548b424b958a8b977367ca4a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.757175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] RBAC: Authorizing `identity:get_group(group_id=d7a1003e548b424b958a8b977367ca4a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.758639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.760545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 167/334] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/groups/d7a1003e548b424b958a8b977367ca4a => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.766291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:08.770772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.814576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc077faa-4f44-4b13-a4c7-7334e2c65272 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.855105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.856012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.856339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.856610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.857503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] RBAC: Adding query filter params (group.id=1e3e6da9920843f293f9b07beea59a05, role.id=215077d32ef84d5eaa4d487616aa2ec3, scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:08.857818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.859358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:08.859957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729839a1-bf3f-4401-969a-509f694076ae None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.868645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 168/335] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/role_assignments?group.id=1e3e6da9920843f293f9b07beea59a05&scope.project.id=b3a6745b8eec4c16b05969f452c0ebe6&role.id=215077d32ef84d5eaa4d487616aa2ec3 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:08.882349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d7bb6-4da7-4e09-88b2-24ebb0a6cc43 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:08.922688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:08.923321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:08.923599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:08.923870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:08.924668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] RBAC: Adding query filter params (group.id=d7a1003e548b424b958a8b977367ca4a, role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=b3acc8f5bcdf4b42bd810be11a69d5c4) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:08.925030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:08.926504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0ab0a-73cc-4fe7-9bc8-b679e1b59c4c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:08.934396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 168/336] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 10 16:50:08 2022] GET /identity/v3/role_assignments?group.id=d7a1003e548b424b958a8b977367ca4a&scope.project.id=b3acc8f5bcdf4b42bd810be11a69d5c4&role.id=6e700d1e090747d6acdf236f6761cbea => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:11.679341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736e63-591b-4fa7-bf1b-eef3e509dd14 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:11.679882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736e63-591b-4fa7-bf1b-eef3e509dd14 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:11.679882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736e63-591b-4fa7-bf1b-eef3e509dd14 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:11.681767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 169/337] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:11.687219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33af92b-1eb4-4b91-bd89-162153fc3e84 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:11.687616 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33af92b-1eb4-4b91-bd89-162153fc3e84 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:11.687784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33af92b-1eb4-4b91-bd89-162153fc3e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:11.712825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e33af92b-1eb4-4b91-bd89-162153fc3e84 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:11.716126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e33af92b-1eb4-4b91-bd89-162153fc3e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:11.811101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 169/338] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:50:12.038200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3f25-ce5b-4ac0-8c4c-024823f28a7b None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.038200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3f25-ce5b-4ac0-8c4c-024823f28a7b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.038200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3f25-ce5b-4ac0-8c4c-024823f28a7b None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.038200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 170/339] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:12.039065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.039065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.039065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.039065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:12.039065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:12.046860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac247bbf-86b3-4763-ad6c-8e0dc8e8d216 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.047139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac247bbf-86b3-4763-ad6c-8e0dc8e8d216 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.047429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac247bbf-86b3-4763-ad6c-8e0dc8e8d216 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.048894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 171/340] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:12.056076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.056076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.056581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.078191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 170/341] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:12.079607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:12.080849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:12.151986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 172/342] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:12.181187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.190367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:12.303818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ced32-52bb-477c-a0fe-6880b16565aa None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:12.343480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:12.344476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.344966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.345441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.361904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:12.363651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:12.384922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:12.385932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:12.386802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:12.386802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:12.386802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:12.388867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 171/343] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:12.394498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.442091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.465727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:12.509146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf1088-9c1b-4b4d-8e98-e3107b4db03f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:12.540123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:12.544861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:12.545599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.546036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.546408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.547157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:12.547589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:12.549147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:12.562217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 173/344] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:12.563231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.576719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295362-4909-4b8f-8277-ec2a9d65ae85 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:12.612380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:12.613007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.613287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.613559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.625781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:12.627244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-86ec4a79-d83e-4ea3-bad7-a84bfc5233cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:12.643389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:12.644353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:12.645220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:12.645220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:12.645220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:12.645220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 172/345] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:12.646799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6cab2-bcda-4324-b8af-ab4ec98572bb None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.648079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6cab2-bcda-4324-b8af-ab4ec98572bb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.648079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6cab2-bcda-4324-b8af-ab4ec98572bb None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.648762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 173/346] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:12.649497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6cab2-bcda-4324-b8af-ab4ec98572bb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.673706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d781109-1bcb-439c-b491-a449ae20fe0e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:12.691604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:12.708336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:12.709487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.709763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.710033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.720293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:12.721730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:12.737333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:12.738271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:12.739293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:12.739293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:12.739293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:12.739293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 174/347] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:12.739293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.762684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6cab2-bcda-4324-b8af-ab4ec98572bb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:12.775433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:12.803482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:12.804124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.804404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.804671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.805318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:12.806999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:12.841388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80e472-e36a-4100-a198-0cdcf651f273 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:12.874922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:12.875536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.875816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.876117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.876760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:12.877061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:12.878418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77036770-604d-4d89-aa23-3cc3df8a2144 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:12.894752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 175/348] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:12.907252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 174/349] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:50:12 2022] POST /identity/v3/users => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:12.907411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:12.907509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:12.907599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:12.907599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:12.917914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:12.919313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:12.964771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:13.016755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e3eaf-aba6-484f-968c-6a6b27d0ba82 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:13.022536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 176/350] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:13.024302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:13.075479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:13.076145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.076438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.076711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.077375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:13.077679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:13.085979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:13.110736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 175/351] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:13.118158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:13.149432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:13.179176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331a7fb-99e8-4fcf-a2f5-6e559ce446da None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:13.220827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab030d4c-964e-433f-9bc0-bbbee5682065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:13.221501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030d4c-964e-433f-9bc0-bbbee5682065 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.221836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030d4c-964e-433f-9bc0-bbbee5682065 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.222134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab030d4c-964e-433f-9bc0-bbbee5682065 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.222780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab030d4c-964e-433f-9bc0-bbbee5682065 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:13.224583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab030d4c-964e-433f-9bc0-bbbee5682065 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:13.232952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3229496-b2f3-4855-afdd-d0019e7d8b3c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:13.270762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6448-5195-4a65-b5e6-196afc620f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:13.272334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6448-5195-4a65-b5e6-196afc620f19 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.272676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6448-5195-4a65-b5e6-196afc620f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.273011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6448-5195-4a65-b5e6-196afc620f19 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.273723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c6448-5195-4a65-b5e6-196afc620f19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:13.275232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c6448-5195-4a65-b5e6-196afc620f19 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:13.312061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 177/352] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:50:13 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:13.313635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3dbd-60fe-47a4-a755-7f4b5d8740c7 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.314083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3dbd-60fe-47a4-a755-7f4b5d8740c7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.314341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3dbd-60fe-47a4-a755-7f4b5d8740c7 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.315768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 178/353] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:13.321518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3dbd-60fe-47a4-a755-7f4b5d8740c7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:13.343586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:13.400135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 176/354] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:50:13 2022] POST /identity/v3/users => generated 248 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:13.400135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.400135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.400135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.438306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:13.439599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:13.521822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 177/355] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:13.525678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3dbd-60fe-47a4-a755-7f4b5d8740c7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:13.563592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:13.564305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.564584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.564854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.577726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:13.579293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:13.595187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:13.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:13.597000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:13.597000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:13.597000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:13.597000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 179/356] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:13.602842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:13.622177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:13.735446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab4136-4deb-4983-b022-32b0f85c2bb9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:13.783457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:13.784127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:13.784350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:13.784627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:13.785274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:13.785580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:13.787488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:13.799565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 178/357] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:13.805368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:13.831133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:13.935359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:13.971919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:13.977701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74ddc-2a60-4ee5-ba23-07f162166c44 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:14.015319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:14.016334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:14.016818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:14.017296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:14.018177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:14.019740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:14.040525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6a37-6c15-4c37-83b6-6d9eb310ba30 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:14.062319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.sql.core [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 16:50:14.062319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 10 16:50:14.062319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [parameters: {'id': 'ad0c25ba212c4dd3911ab49cd5d1e7ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 10 16:50:14.062319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81514) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result.read() Oct 10 16:50:14.080363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return ref.to_dict() Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:50:14.081255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application yield resource Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise self.value Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application session.commit() Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 16:50:14.082183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.session.flush() Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application compat.raise_( Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 16:50:14.083014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:50:14.084615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result.read() Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:50:14.085481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [parameters: {'id': 'ad0c25ba212c4dd3911ab49cd5d1e7ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:14.086226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 10 16:50:14.087016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:14.091146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 179/358] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 10 16:50:13 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:14.097153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:14.120080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:14.121045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:14.123177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:14.123647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:14.124498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:14.127347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:14.227734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc11ab-5b17-40a0-95e0-b1d4b2923b31 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:14.254884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 180/359] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:50:13 2022] POST /identity/v3/users => generated 249 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:14.273103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac78d16-9acb-4ddd-b558-c956b53157fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:14.274083 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac78d16-9acb-4ddd-b558-c956b53157fa None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:14.274542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac78d16-9acb-4ddd-b558-c956b53157fa None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:14.275026 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac78d16-9acb-4ddd-b558-c956b53157fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:14.278734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:14.295783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac78d16-9acb-4ddd-b558-c956b53157fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-9ac78d16-9acb-4ddd-b558-c956b53157fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:14.301685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:14.302681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 16:50:14.303448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:14.311087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 180/360] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:14.317231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:14.465691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83f2f9-e579-4dec-94b5-98c6c27a1ca5 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:14.498559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:14.501890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:14.502513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:14.503021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:14.503388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:14.504131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:14.504512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:14.506544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83160b-ff65-4162-9caf-008aee44ac50 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:14.519823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 181/361] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:15.407471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c53ace-3d04-44a3-aba8-f2212e4d3023 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:15.408535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c53ace-3d04-44a3-aba8-f2212e4d3023 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:15.409025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c53ace-3d04-44a3-aba8-f2212e4d3023 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:15.410853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 181/362] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:15.416903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0ab35-af84-4c63-ac54-fef7023ccccf None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:15.417724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0ab35-af84-4c63-ac54-fef7023ccccf None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:15.418194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0ab35-af84-4c63-ac54-fef7023ccccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:15.437964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7ed0ab35-af84-4c63-ac54-fef7023ccccf None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:15.439454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7ed0ab35-af84-4c63-ac54-fef7023ccccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:15.512342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 182/363] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:15.803135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c53ace-3d04-44a3-aba8-f2212e4d3023 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:15.924629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d764ae-3576-4e6b-8e95-06fe472403b2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:15.924955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d764ae-3576-4e6b-8e95-06fe472403b2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:15.925186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d764ae-3576-4e6b-8e95-06fe472403b2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:15.926801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 183/364] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:15.932167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d2428-fb20-42ae-b80e-aa0d150b978d None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:15.932452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d2428-fb20-42ae-b80e-aa0d150b978d None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:15.932731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d2428-fb20-42ae-b80e-aa0d150b978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:15.938513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c53ace-3d04-44a3-aba8-f2212e4d3023 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:15.952333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a21d2428-fb20-42ae-b80e-aa0d150b978d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:15.953564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a21d2428-fb20-42ae-b80e-aa0d150b978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:15.977020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:15.978035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:15.978513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:15.979067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:15.991810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:15.997713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:15.998682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:15.999506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:16.007561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 182/365] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:16.009323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.027946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 184/366] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:50:16.072872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:16.116623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1989f-998e-4205-bce1-b9fbd4ad5a24 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.116943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1989f-998e-4205-bce1-b9fbd4ad5a24 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.117169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1989f-998e-4205-bce1-b9fbd4ad5a24 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.118744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 185/367] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:16.122340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b265-62b1-40fe-82f8-3ea8059e486e None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:16.123662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.123938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.124256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.152860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:16.154005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:16.168237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:16.169202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.169678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.170143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.171022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:16.171663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:16.175182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:16.188360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 183/368] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:16.200313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.247599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:16.290454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 186/369] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:50:16.365358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.374398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20766ba5-ad0b-45ea-b1b2-fa03454320d0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:16.412382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:16.423236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:16.423872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.424183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.424459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.436130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:16.437590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:16.452851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:16.453748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:16.454748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:16.454748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:16.454748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:16.454748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 184/370] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:16.460940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.523032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fd1b8-1a60-4aef-8a60-14b22ba50d62 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:16.577513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100ab7e-c78a-4713-a573-dcb96297dbf9 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:16.612768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:16.613266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:16.613602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.613954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.614318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.615271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.615773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.616286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.617195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:16.617696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:16.619303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:16.637154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 185/371] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:16.639749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.649787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:16.659125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:16.660051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:16.660850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:16.666875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 187/372] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:16.673944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.700830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:16.720074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:16.777694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9655f44-db23-4c48-a0d2-3d6ec86f026d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:16.799930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c5ff6-4c03-4d1a-b92f-353054487fd8 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:16.824735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:16.825365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.825641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.825905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.837826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:16.839834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:16.847324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:16.847941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:16.848314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:16.848582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:16.849777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:16.850075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:16.851437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:16.863249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 188/373] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:16.863249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:16.865583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:16.873258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 186/374] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:16.874165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:16.998380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.002583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d1d9-21ff-4ce2-948e-7b9f7070acf2 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.006834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.016977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcefa1-cabf-4b10-9e22-89bdb1694a66 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.040192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.040946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.041324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.041737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.053573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:17.059105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:17.060106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:17.060908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 189/375] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:17.061563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.066369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.067017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.067297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.067567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.078707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.079857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-c38e4c11-fd9a-4bd0-9b61-28026cc7b8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:17.098415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:17.099621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:17.100617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:17.100617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:17.100617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:17.100617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 187/376] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:17.105356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabbc29-2d7e-4f4a-b274-c1f1d1f4a517 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.105648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabbc29-2d7e-4f4a-b274-c1f1d1f4a517 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.105920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabbc29-2d7e-4f4a-b274-c1f1d1f4a517 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.107211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 188/377] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:17.111430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabbc29-2d7e-4f4a-b274-c1f1d1f4a517 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.175303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b95ef-e061-48b0-a747-4891dfdc3211 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.214911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.215520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.215798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.216090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.216813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:17.217115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.218535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.234395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabbc29-2d7e-4f4a-b274-c1f1d1f4a517 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.256173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:17.257011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.276482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.279737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.280384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.280672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.280941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.281613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:17.281919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.286847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db6d7-cfc2-43f1-9722-576602e2c76f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.300478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:17.300478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.300478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.300478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.307877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.319926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:17.321185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:17.374956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034d1fc-4a42-460e-9a5d-2847ea489806 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.402137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 190/380] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:17.402989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.419936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.420990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.421465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.421954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.422798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:17.423296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.425756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.443430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 191/381] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:17.444771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.546388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f743f47-4463-4bdc-a219-242e47b6d5f0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.547847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.576942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe80d4-8e1d-4875-950b-61fa576d3233 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.586837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.587690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.587962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.588254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.590265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.600460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.602218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.620642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.621289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.621572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.621845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:17.624123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:17.625014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:17.625830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:17.625830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:17.625830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:17.625830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 191/382] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:17.626050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.638145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.640365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-555bf4bb-d0a0-466c-b196-0bd6106e0ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:17.657565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:17.658442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:17.659268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:17.659268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:17.659268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:17.659268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 192/383] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:17.659999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b56f00-a1f8-494f-b631-1ab9e06dcb0e None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.659999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b56f00-a1f8-494f-b631-1ab9e06dcb0e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.660304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b56f00-a1f8-494f-b631-1ab9e06dcb0e None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.661752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 193/384] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:17.662285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b56f00-a1f8-494f-b631-1ab9e06dcb0e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.737910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e74cde-b7e7-4e79-b7c9-a8622f7d5ea7 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.766232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b56f00-a1f8-494f-b631-1ab9e06dcb0e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.772108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.772726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.772997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.773261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.780505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.781945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.791953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:17.792722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:17.793703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 192/385] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:17.794569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.806736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.807354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.807628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.810333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.810333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:17.810333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.810333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8964688-b335-47a4-8fd7-53163a799b14 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.822372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 194/386] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:17.823564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.823847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.824160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.842186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:17.843376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:17.845516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:17.899302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b7f7f-6816-4033-9ce6-fbb686af4222 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:17.909156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 195/387] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:17.910046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:17.931791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:17.932426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:17.932704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:17.932973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:17.933604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:17.933905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:17.935248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:17.947266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 193/388] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:17.948148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.021642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6374-ae0e-4396-bef7-729c2b466965 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.038824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.048151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bf49c-919c-42c3-864f-a4081457d918 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.056243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.057429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.058309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.058593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.058863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.071770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:18.079275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:18.081413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 196/389] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:18.082202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.086988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.087654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.087941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.088239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.088981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:18.089289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.090725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.129296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 194/390] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 10 16:50:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:18.132404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.198962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19d76-1dfb-4bcf-92b8-fce64af4cdbe None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.232265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baecbf5-aad4-48c0-98e9-010dc1107950 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.232608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.233238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.233520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.233796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.241041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.242493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:18.253812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:18.254825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 197/391] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:18.255703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.269931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.270559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.270987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.271260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.278398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.279774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:18.291543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:18.292617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:18.293564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 195/392] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:18.294170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.297933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.318159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.353564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdd71f-980c-4a04-83b5-bcbaf34da316 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.386255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.386859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.387137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.387406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.388089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:18.388392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.389762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.390763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10344f-b094-4e6b-915e-e4f2900e1b9d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.397442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 198/393] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:18.398217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.426733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.428365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.428719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.429064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.438960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.440440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:18.454801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:18.455863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:18.456943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:18.456943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:18.456943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:18.456943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 196/394] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:18.456943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.495398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.498171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04618e2f-4aac-4a39-ae8c-cf9f153809da None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.533749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.534276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.534549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.534817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.537830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.546194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.547653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.561165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fc66b-69e3-4eae-834a-0cf17e694dce None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:18.564003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:18.565340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:18.566192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:18.566192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:18.566192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:18.566192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 199/395] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:18.566402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.595200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.595804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.596119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.596390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.597085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:18.597387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.598809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.616299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 197/396] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:18.617081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.669644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfbe16-a15d-4558-ae9d-5390a7eebeb9 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.702338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.702953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.703230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.703503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.704256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:18.708849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.708849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.723406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a913e-7928-44d9-81be-0bad8ac40535 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.724637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 200/397] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:18.725464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.736879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.757203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.757824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.758096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.758361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.760962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.768779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.770342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:18.784707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:18.785911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:18.787531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:18.787531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:18.787531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:18.787531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 198/398] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:18.789521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.828215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e24379-aa7a-41e3-ab1a-b901038b479a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.861864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.862472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.862751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.863525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.863656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:18.863958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.865401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.876824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 201/399] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:18.877599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.887649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f52c4-baca-4ca2-9b25-6c1676833fec None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.921459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:18.922061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:18.922336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:18.922610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:18.923258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:18.923564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:18.924940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:18.936391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 199/400] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:18.937164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:18.949917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:18.980199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab9e5a-0848-45ed-927d-1e826ae10478 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:18.984528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.013396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.014181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.014465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.014738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/54179215880d4ed291d18944a2f86fdc/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.029139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=54179215880d4ed291d18944a2f86fdc, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.030581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.034299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f6e9-2491-4bd4-9b39-e26f3198d5c5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.066645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.067228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.067508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.068339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.078563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.079987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:19.094457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:19.095564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:19.101224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:19.101224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:19.101224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:19.101224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 200/401] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:19.101224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.130735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 202/402] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:18 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/54179215880d4ed291d18944a2f86fdc/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:19.131617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.198660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.201450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e97078-4998-49af-b50b-b00a7228d4d8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.242130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.242753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.243033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.243306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.243967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:19.244303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.245711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.258228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 201/403] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:19.259085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.260132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2907-cf7c-4d3b-b142-c1f9aa0687e1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.285456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.300940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.301592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.301871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.302686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.310886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.312383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:19.325030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:19.325882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:19.326716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 203/404] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:19.327858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.377658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e57790-c3f3-4d4e-bfce-2d3de0851802 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.414301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.415217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.415678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.415945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.416619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.418027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.434350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11566a-2317-41b4-832e-5c128dc50206 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.467444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.468076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.468354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.468620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.469252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:19.469545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.470885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.482273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 204/405] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:19.483049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.500059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 202/406] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:50:19 2022] POST /identity/v3/users => generated 247 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:19.500059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.526869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.533793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.596167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e16494-5993-4098-9a31-c4332d7c3190 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.637104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.638079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.638360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.638631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.646488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3488-509c-4f90-ab04-e8a99ed324d6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.650233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.651682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:19.669628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:19.670597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 16:50:19.671481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:19.672007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 205/407] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:19.673296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.688634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.689414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.689922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.690200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.690904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:19.691210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.693980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.707605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 203/408] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:19.714362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.792851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.797897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993aedba-f7f4-4510-af78-99266166fa3e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.800380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:19.827802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd25d-82a6-4132-91ff-f845f4043103 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:19.835769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.836562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.838955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.838955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.844508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.846084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:19.858582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:19.859625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:19.860714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:19.872305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:19.872305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:19.872305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:19.872305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:19.872742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:19.872825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 206/409] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:19.872917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:19.873000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.873085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:19.888755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 204/410] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:19.889614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:19.995560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba2f82-dae6-4d65-83e4-9a8c81c44876 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.001217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa1e43-64b1-4c1d-923b-8b7414ebe58b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.030781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.031491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.031767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.032090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/dfe6839ce3ac4e6eb27b54eb1870c67d/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.049480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=dfe6839ce3ac4e6eb27b54eb1870c67d, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.051005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.052679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.053574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.054423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.054908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.056844 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:20.057209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.058782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.060736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:20.071310 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 205/411] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:20.072550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:20.130695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 207/412] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:19 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/dfe6839ce3ac4e6eb27b54eb1870c67d/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:20.130695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:20.153261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:20.237565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8777d-8312-472e-b3ab-d5fa17d31b55 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.305063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.305974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.306249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.306514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.324604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.326088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.332941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a952cc7-b058-4b1c-869c-065268ed8165 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:20.355303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:20.356290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:20.357866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:20.357866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:20.357866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:20.357866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 206/413] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:20.365064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:20.391277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.391876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.392183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.393882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/eff532d7c1bc4c02838873c67fca787f/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.413519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=eff532d7c1bc4c02838873c67fca787f, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.414952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.427366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:20.501462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d74ff-288a-4daf-ab85-4643699906c4 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.528898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 208/414] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:20 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/eff532d7c1bc4c02838873c67fca787f/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:20.542454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.544135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.544422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.544700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.545520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:20.545833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.547295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.562077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 207/415] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:20.568568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:20.651123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:20.711133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ede5-1ffc-4f4f-ac24-c4029b1b9660 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.751801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.752474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.752756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.753076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.760647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.762086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:20.774155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:20.774991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:20.775835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 209/416] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:20.781242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:20.935299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3a8ba-e0d5-4a4e-99f2-961863cdccf2 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:20.949453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:20.979428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:20.980053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:20.980346 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:20.980620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:20.981320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:20.981627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:20.983046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe59ed-4c77-4860-9fe0-dd61940e330c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:20.995414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 208/417] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:21.001073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:21.126729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a059d-4a44-49b6-87d7-45bf16e1135b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:21.155362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:21.173847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6059a0-26db-4d2c-9290-44ffdfa62402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:21.174577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6059a0-26db-4d2c-9290-44ffdfa62402 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:21.174915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6059a0-26db-4d2c-9290-44ffdfa62402 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:21.175194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6059a0-26db-4d2c-9290-44ffdfa62402 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/837a59258e6c43c8befebc2639ab16f4/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:21.196614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6059a0-26db-4d2c-9290-44ffdfa62402 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=837a59258e6c43c8befebc2639ab16f4, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:21.197972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6059a0-26db-4d2c-9290-44ffdfa62402 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:21.279126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 210/418] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:21 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/837a59258e6c43c8befebc2639ab16f4/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:50:21.506814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9414c-aa9e-4dfc-8f5d-1419b3065a4d None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:21.506814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9414c-aa9e-4dfc-8f5d-1419b3065a4d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:21.506814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9414c-aa9e-4dfc-8f5d-1419b3065a4d None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:21.506814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 209/419] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:21.512930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e668-4c54-4896-815f-d561623658f1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:21.513458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e668-4c54-4896-815f-d561623658f1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:21.513934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e668-4c54-4896-815f-d561623658f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:21.532265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d301e668-4c54-4896-815f-d561623658f1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:21.533776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d301e668-4c54-4896-815f-d561623658f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:21.605615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 211/420] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:21.871109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33511196-617b-4d12-84c2-9d717fa66138 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:21.871537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33511196-617b-4d12-84c2-9d717fa66138 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:21.871635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33511196-617b-4d12-84c2-9d717fa66138 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:21.873238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 210/421] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:21.878487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:21.879279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:21.879751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:21.908973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:21.910511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:21.941081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33511196-617b-4d12-84c2-9d717fa66138 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:21.988462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 212/422] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:22.079431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33511196-617b-4d12-84c2-9d717fa66138 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:22.115005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:22.115966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:22.116563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:22.117036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:22.132316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:22.137550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:22.138446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:22.139226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:22.141623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 211/423] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:22.149168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.188647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:22.294742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499cde-746e-48fc-a87e-ebaf1707f0f5 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:22.333607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.355117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:22.355117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:22.355117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:22.355117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:22.355528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:22.355528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:22.355528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:22.372508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 213/424] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:22.381587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:22.382524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.488414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:22.495404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f08618-7385-49c9-b880-cd4427b9bc82 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:22.512118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b625a8-4c91-42a7-b2a5-550f3239ee08 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:22.554431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:22.555074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:22.555345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:22.555876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:22.571501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:22.572481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:22.572757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:22.573026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:22.577276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:22.582006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:22.582883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:22.584240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 212/425] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:22.591023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.606168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:22.607617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:22.634756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:22.635682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:22.636568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:22.636568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:22.636568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:22.640119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 214/426] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:22.641169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.702462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:22.721863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750cea-9d3e-4c31-acbb-23c7ef9d3fbe None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:22.781735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bcbe6-0964-4239-b9b7-041f5a8ae805 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:22.800760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:22.801380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:22.801687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:22.801948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:22.802630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:22.802933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:22.804320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:22.820568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:22.821504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:22.822011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:22.822488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:22.823330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:22.823834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:22.825415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:22.828427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 213/427] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:22.834410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.840135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 215/428] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:22.845136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:22.861988 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:22.982500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c15f0-eb2b-48e2-a64d-62c5ec2f8ae6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.018193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.018827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.019105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.019378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.031138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.032626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.043246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.048438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59357b6-3526-40b1-89ef-4f8a7d9699d5 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-3b18caed-2fcc-43d3-9eb0-066660176d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:23.051062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:23.052731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:23.053766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:23.053766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:23.053766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:23.055510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 214/429] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:23.061643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da632c-07bf-4e38-bb25-bd2712ebec50 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.062154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da632c-07bf-4e38-bb25-bd2712ebec50 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.062603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da632c-07bf-4e38-bb25-bd2712ebec50 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.064092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 215/430] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:23.064957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da632c-07bf-4e38-bb25-bd2712ebec50 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.099899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.100563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.100839 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.101105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.109844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.111205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-3754ea8f-4e15-4c12-93ce-8e6291b461b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:23.123432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:23.124411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:23.125289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 216/431] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 10 16:50:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:23.126324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.126485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.126726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.149491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:23.150669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:23.200157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da632c-07bf-4e38-bb25-bd2712ebec50 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.231637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.236013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.236763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.237289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.237625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.238401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:23.238679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.240682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.248751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 217/432] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:23.249524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.256959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 216/433] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:23.262585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.366894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.410200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85ba52-41de-476d-bc5b-54754b9823ea None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.435930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a95d-4235-4cfe-907e-4f9a4e3bfd99 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.447822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.450107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.450391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.450662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.460813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.462324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-0e8256ab-712a-44d0-abba-d72d6ff3051e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:23.474509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:23.475426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:23.476342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 217/434] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:23.476887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2496-731b-49b9-8f81-25cde117d31a None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2496-731b-49b9-8f81-25cde117d31a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.477432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2496-731b-49b9-8f81-25cde117d31a None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.478759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 218/435] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:23.483465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2496-731b-49b9-8f81-25cde117d31a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.485497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.486536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.487141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.487483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.488343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:23.488766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.490892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d170a11-ffa9-42fa-9ac1-e785018ce809 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.510086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 218/436] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:23.511386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.511744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.512167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.531129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:23.532647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:23.583622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.616059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 219/437] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:23.616059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.618863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2496-731b-49b9-8f81-25cde117d31a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.641932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.657209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.658106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.658554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.658994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.659990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:23.660535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.662824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.682635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 219/438] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:23.683916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.733637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3415e0-ac1e-44fc-aad9-5d82b6660aa7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.787343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcbdf5-c5d4-42f3-aad2-4598c726e767 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.798107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.798733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.799017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.799292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.828976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.830115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.830586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.831071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:23.844928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:23.850930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:23.851828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:23.852656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:23.855559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 220/439] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:23.856720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.862840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.884313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:23.886581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:23.903316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:23.963313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f46b6a-d671-46a3-9ad5-223eb5d33ed6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:23.987603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 220/440] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:23.988470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:23.996883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:23.997507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:23.997781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:23.998050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.008190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.009605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:24.024257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.025139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:24.025965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:24.025965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.025965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:24.025965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 221/441] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.026188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.067997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.085363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027d71-8a1e-495c-8b7c-40045b760821 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.116639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.118391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.118992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.119271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.119541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.120226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:24.120530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.121891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c09a9e-b2f2-4500-8562-1c0d6212a65e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.125590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa8f5e-0056-41ab-809a-565d3db7efdc None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.129592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 221/442] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:24.129592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34ea3-a66c-4041-9d5e-3fc799eecc81 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.129592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34ea3-a66c-4041-9d5e-3fc799eecc81 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.129592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34ea3-a66c-4041-9d5e-3fc799eecc81 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.129592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 222/443] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:24.129592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34ea3-a66c-4041-9d5e-3fc799eecc81 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.159909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.160583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.160863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.161133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.171781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:24.176774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.177709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:24.178494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 222/444] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.179152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.237373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34ea3-a66c-4041-9d5e-3fc799eecc81 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.270366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.271072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.271352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.271622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.274773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e61dd-a050-4533-8724-428c28df439c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.282026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.283528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.286070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:24.298784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.299835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:24.300887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:24.300887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.300887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:24.304173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 223/445] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.305124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.308797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.309404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.309683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.309954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.310593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:24.310894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.312281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045c5be-4ccc-42b8-905a-e8a62ec480e4 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.327970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 223/446] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:24.327970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.327970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.327970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.342112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:24.343319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:24.346471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.403851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9fabc-b9d6-4ac7-a05d-88ec5e04e286 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.408087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 224/447] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:24.408906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.441401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.442032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.442312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.442613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.453304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.454743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:24.469153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.470058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:24.471478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:24.471478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.471478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:24.471478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 224/448] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.471478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.520913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332f123-3739-47d1-92c9-a128543032f3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.523514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.564909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.565512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.565786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.566054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.566770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:24.567078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.568524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.581263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 225/449] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:24.581263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.581263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.584417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d638f-89b7-4c27-b0b2-ae2bc9bc55bf None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.620343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a242a-9387-400f-a540-0a887e076442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.620969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a242a-9387-400f-a540-0a887e076442 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.621253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a242a-9387-400f-a540-0a887e076442 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.621525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a242a-9387-400f-a540-0a887e076442 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.622166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a242a-9387-400f-a540-0a887e076442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:24.622472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a242a-9387-400f-a540-0a887e076442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.623846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a242a-9387-400f-a540-0a887e076442 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.640808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 225/450] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:24.640808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a242a-9387-400f-a540-0a887e076442 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.688985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d661f7c-d3eb-48d5-a256-751f84578dbd None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.722002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.722627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.722907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.723172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.730354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.731781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.737884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a242a-9387-400f-a540-0a887e076442 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:24.743537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.744520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 226/451] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.745544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.787856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.811990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.812796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.813071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.813338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.813969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:24.814268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.815615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.827174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 226/452] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:24.827942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.841985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:24.880784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47f80-a363-434c-9f38-d6cf4c2e2081 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.913837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.914454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.914730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.914998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.925392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.926836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.926947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee32f5-4c4a-483b-abe2-81976073495b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:24.940778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.941829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:24.943197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:24.943197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:24.943197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:24.943197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 227/453] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.943197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:24.965548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:24.966562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:24.966851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:24.967124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:24.973654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:24.975064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:24.985858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:24.986860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 227/454] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:24.987881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.015505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.047990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abbb0d-5dcc-4a11-a11c-048d40626e47 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.061936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.081800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.082418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.082696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.082968 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.085491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97223ae9-de11-4581-ad50-37aafa800947 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.093860 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:25.098857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:25.099896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 228/455] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:25.105016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.118740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.119345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.119629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.119906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.120644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:25.120956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.122359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.140332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 228/456] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:25.140929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.199811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5801e0a-d0ba-426e-be47-7ed51b98d37b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.224848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.233546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.237504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.237504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.237504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.237743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffedfe-2d9e-40f5-80fc-2210186ee7e0 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.241886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.243358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:25.254221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:25.255802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 229/457] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:25.256723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.272318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.272932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.273211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.273478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.274110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:25.274411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.275760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.286809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 229/458] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:25.287582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.295930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.359205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3572e-fde5-4e0c-9c0b-98fab6bf47f9 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.386417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40826d-523b-4fb2-a2c7-d534b2562191 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.392971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.393572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.393872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.394127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.394845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:25.395151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.396651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.407711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 230/459] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:25.408508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.423144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.423745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.424033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.424310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.424956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:25.425252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.426604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.435001 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 230/460] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:25.435761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.447987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.504186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.510814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad849cd-34c6-4994-a80d-93553ca1ab2a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.531295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf601d-7b3e-45a0-8fa6-32589d819a0d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.543150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.543802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.544109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.544393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c3e348fd8e2b4203b7bc43240cb79d74/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.559015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=c3e348fd8e2b4203b7bc43240cb79d74, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.560488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.564225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.564858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.565139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.565408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.566118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:25.566420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.568421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.584778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 231/461] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:25.585546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.621574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 231/462] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:25 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c3e348fd8e2b4203b7bc43240cb79d74/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:25.622377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.656191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.694946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdbcd9-3077-41fe-b3a1-a5951b3f2659 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.734010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.734757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.735025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.735296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.737714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e8805-26f8-4f49-9b51-634e5b18c805 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.743695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.745197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:25.757698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:25.758673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:25.759557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 232/463] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:25.764494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.781714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.782435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.782822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.782992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.783907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=54179215880d4ed291d18944a2f86fdc) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:25.784470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.785929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:25.794517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 232/464] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/role_assignments?user.id=54179215880d4ed291d18944a2f86fdc&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:25.794841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:25.807215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.898700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de40a52-83f1-4dc3-bcc0-a96272761823 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.929109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:25.945444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:25.946127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:25.946404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:25.946682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:25.947124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f724ec-ac12-46f3-b085-9c424c114337 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:25.969497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:25.970997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:26.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:26.005635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:26.006464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:26.006464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:26.006464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:26.006464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 233/465] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:26.006684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.017384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.018016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.018292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.018563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.100041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.101509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.111975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.142268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707adfd-998b-4f1f-86b7-edaacbef41d7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.207103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.207784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 233/466] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:26.211552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.212413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.213266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.213664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.214971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:26.216605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.218632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.254346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 234/467] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:26.255617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.315129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.319922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c29e63-306f-48a7-8ffb-7210b47a9368 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.330783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.361074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.362338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.362807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.363238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.364073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:26.364925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.366526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.388466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 234/468] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:26.389755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.395451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cc24d-5ac0-40d1-98e7-1d67fd9b62e1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.434434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.434434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.434733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.434892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.449037 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.450793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:26.468706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:26.469751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:26.475399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:26.475399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:26.475399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:26.475399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 235/469] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:26.475399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.579222 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.584251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be72fe4-5a9d-47ef-8205-9c7c89333ea8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.591880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819b163-6f1f-4122-b3f4-11bc6ec8d128 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.620217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.621151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.621647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.622140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.630833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.631469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.631751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.632051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.640507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.642116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:26.655225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:26.656219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 16:50:26.657103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:26.661320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 236/470] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:26.662183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.665429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.701995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.703618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.773952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c08f5-0586-44b0-bfde-bbc2ce78de51 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.809548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.810532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.811016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.811488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.812576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:26.813135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.813410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 235/471] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:26.814302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.815188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:26.828895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 237/472] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:26.830036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:26.842369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.912454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:26.946300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d419d-ffeb-4615-a050-b012e42d2d40 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.952702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d34a9f-399f-4411-a244-f6a3759664ff None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:26.983301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.984356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.984950 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.985422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.996744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.996968 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:26.997565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:26.997849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:26.998121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:26.998855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:26.999162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:26.999764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.000782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:27.017042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:27.017954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:27.018786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:27.018786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:27.018786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:27.020566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 238/473] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:27.021659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.024267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 236/474] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Oct 10 16:50:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:27.026222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.108369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.136715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b8f7-38b2-4e9c-aa7d-5f2544dfbce6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.151549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d79802-066c-49a9-9e57-7b7e223492aa None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.168453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.179655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.180308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.181150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.181434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.189482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.190349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.190846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.191905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.199630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.201687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:27.219554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:27.220498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:27.221312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:27.224505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 239/475] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:27.225344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.262985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.265042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.358104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a6284-6e82-47b1-a0df-2615cc8b3ffc None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.374815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 237/476] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:27.375948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.389120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.403913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.405755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.406770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.407194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.407736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.408897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:27.409653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.411699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.428279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 240/477] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:27.429470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.480305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045c1fd-15e5-4564-a93d-a59fba5e32c3 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.526340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.527205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.527885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.528353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.529943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:27.530427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.532628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.548497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 238/478] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:27.552548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e4a1f-de21-4e23-92c3-0b9c2671850d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.555184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.596472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.597833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.598319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.599362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.612371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.613904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:27.628844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:27.629998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:27.631030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:27.631030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:27.631030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:27.632443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 241/479] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:27.633472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.658600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.675978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.680326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1f79-6516-4cc4-849f-d5544474deb6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.728153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.729174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.729895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.730331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.741280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.743865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.754065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc70824-0f58-491e-9917-51133b666a84 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:27.760923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:27.762037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:27.763080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:27.765836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 239/480] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:27.766820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.800142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.800867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.801292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.801643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.802413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=dfe6839ce3ac4e6eb27b54eb1870c67d) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:27.802796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.804611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.820054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 242/481] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/role_assignments?user.id=dfe6839ce3ac4e6eb27b54eb1870c67d&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:27.820054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.918501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5261c-19ef-4533-990a-3f4812cb3489 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.956311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:27.957074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.957354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.957627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.958884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:27.959193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:27.960650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32b69-0eef-419c-8cfc-7626170d1694 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:27.974527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 240/482] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:27.975832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32169e99-6bd1-4fde-87b2-aa45cf97633f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:27.977793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:27.980598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32169e99-6bd1-4fde-87b2-aa45cf97633f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:27.980958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32169e99-6bd1-4fde-87b2-aa45cf97633f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:27.982494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2c46d-fd5c-48d8-9180-a470fc941954 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:27.982675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 241/483] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:27.983445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32169e99-6bd1-4fde-87b2-aa45cf97633f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:27.995850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:28.036414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.037031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.037312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.037584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.038285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:28.038753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:28.040248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f0e8d-bbd1-41ab-b908-50bb03e90e6a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:28.053358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 243/484] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:28.054610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.054897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.055172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.075331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:28.076620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:28.118461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32169e99-6bd1-4fde-87b2-aa45cf97633f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:28.162445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 244/485] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:28.162945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.163322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:28.163567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.163844 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.164172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.186348 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:28.187811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:28.206719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:28.207579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:28.209104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:28.215565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 242/486] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:28.216465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:28.287725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:28.300012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92de3d-9b43-4061-a216-a3a86d19eb48 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:28.302528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:28.338429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.339043 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.339431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.339708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.340471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=eff532d7c1bc4c02838873c67fca787f) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:28.347148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:28.347261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a743e6-5be2-47c5-a3db-ab85cb8c2153 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:28.347339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031d97c-126f-49fa-91d3-480763936115 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:28.356054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 245/487] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/role_assignments?user.id=eff532d7c1bc4c02838873c67fca787f&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:28.356054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554d752-a4f7-406b-93e9-05edfb0dc749 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.356054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554d752-a4f7-406b-93e9-05edfb0dc749 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.356054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554d752-a4f7-406b-93e9-05edfb0dc749 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.356054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 246/488] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:28.370468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.370649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.370931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.400274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:28.401475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:28.423063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.423684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.423958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.424270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.425068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:28.425376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:28.427451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:28.441880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 243/489] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:28.448272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:28.524602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 247/490] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:28.525737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:28.598669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93b2a5-453a-4975-bc98-a8b344a2d02a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:28.636724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:28.650178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.650807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.651603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.651883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.652933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=837a59258e6c43c8befebc2639ab16f4) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:28.653248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:28.655213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:28.656307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:28.663871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 244/491] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/role_assignments?user.id=837a59258e6c43c8befebc2639ab16f4&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:28.679829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4add54-2aad-4e94-b375-d99b0970c7d9 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:28.721125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.721785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.722061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.722333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.734939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:28.740658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:28.741572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:28.742350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 248/492] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:28.749652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:28.867067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068bdd-6167-4f02-ad25-02662aba76fb None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:28.880780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:28.928943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:28.929955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:28.930507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:28.931219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:28.932562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:28.933095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:28.934641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:28.946027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 245/493] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:28.951994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.006075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcb782-b4d5-4117-8eed-97ad53b40dc6 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.010916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.041142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.041751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.042024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.042292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.054331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:29.059880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:29.061126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:29.061927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:29.062537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 249/494] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:29.067677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.068133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c006141-eab5-43bb-b9db-e5f42236b966 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.071421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.105198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.105864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.106143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.106416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.118670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.120141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:29.134794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:29.136087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:29.136948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:29.136948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:29.136948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:29.136948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 246/495] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:29.141724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.173740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa46f75-aefe-492d-9ebb-74b983e7745e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.208712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.209389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.209730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.210064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.210784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:29.211180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.212642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.220869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 250/496] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:29.227102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.248406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81887d40-a112-492e-bac0-bb67edaeeaff None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.250791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.285484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.286148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.288296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.288570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.289224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:29.289528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.290899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.303384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 247/497] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:29.303703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.309050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.329023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddbff8-b3f0-45f8-886c-8f4afdace80d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.364002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.364860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.365190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.365538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.376483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.378022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:29.392441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:29.393416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:29.394275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:29.394275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:29.394275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:29.394275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 251/498] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:29.399968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.416511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122276e-2d37-4050-9c7e-19a5141e2864 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.452817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.453449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.453727 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.453999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.462813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.464251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:29.475941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:29.476912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:29.477890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 248/499] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:29.483019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.502925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f3d78-1c93-4d72-afad-ae32819ccd7b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.510539 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.532423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.564774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.565391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.565670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.566000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.566629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:29.566930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.572201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.598431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 252/500] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:29.604013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.616738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cfb34-9802-43d0-a21a-56ab2850dd0d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.712763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.713429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.713638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.713911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.714647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:29.714949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.716398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.726321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867faf6d-94c8-4a27-bf40-5319cde47a5c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:29.760192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 249/501] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:29.762693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.802967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:29.803608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:29.803883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:29.804185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:29.819044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:29.822888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:29.847721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:29.848859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:29.849803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:29.853225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 253/502] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:29.861042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:29.924424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.931192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:29.963530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d62147-4092-4853-b1b5-456be6e9520a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.020252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.020901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.021181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.021456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.055282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c294d-c977-4bb5-8738-95b306a06b85 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.092862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.094307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.120722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.121341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.121621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.121892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.122676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:30.122982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.124410 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.146022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 254/503] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:30.151409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.176447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:30.196520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 250/504] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 16:50:30.213510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.274347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:30.277317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e20a77-d275-4bcb-bb69-c3670a9e7574 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.334126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.334761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.335370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.335370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.355507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.356965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:30.390221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:30.391409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:30.392325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:30.392325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:30.392325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:30.392325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 255/505] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:30.399849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.432783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca41bf-27b0-4198-8c51-013bb840423f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.508735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.509381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.509663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.509938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.537745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.539131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.544202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37824eb8-5f75-4394-a5f6-46d5ae89b364 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:30.578656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:30.579687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:30.580603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:30.580603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:30.580603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:30.585851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.586472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.588107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.588107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.588107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:30.588107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.588533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 251/506] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:30.594171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.601106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.608577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 256/507] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:30.614953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.624300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:30.704163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:30.730022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fdbe3-0c35-4acc-8366-e54bccff5044 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.776293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.777194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.777476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.777750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.787961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.789371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.793504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034bf57-5e58-4d5b-be3a-0ed0e97af293 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-9fe4d681-d033-4d17-9131-7d9abfca872d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:30.809600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:30.810519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:30.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:30.818541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 257/508] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:30.818541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ee01-1fa2-4cbe-a110-ed70634ac650 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.818541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ee01-1fa2-4cbe-a110-ed70634ac650 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.818541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819ee01-1fa2-4cbe-a110-ed70634ac650 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.819681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 258/509] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:30.820278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819ee01-1fa2-4cbe-a110-ed70634ac650 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.838287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.839282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.839756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.840267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.841111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:30.841609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.843196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22e137-7b13-4892-b461-2707a51859aa None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:30.861694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 252/510] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:30.861694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.861974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.862619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.890443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:30.891986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:30.906095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:30.943148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819ee01-1fa2-4cbe-a110-ed70634ac650 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:30.968645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 253/511] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:30.969837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:30.991475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:30.996074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:30.996074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:30.996074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:30.996074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:30.996074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:30.996074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2a3ab-c639-4def-923e-c14de4ab393e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.000019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.008257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 259/512] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:31.011580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782e582-9604-4287-bd3b-567c464e4237 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.011580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782e582-9604-4287-bd3b-567c464e4237 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.011580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782e582-9604-4287-bd3b-567c464e4237 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.011580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 260/513] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:31.013466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782e582-9604-4287-bd3b-567c464e4237 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.087983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9d43e-81ff-4d1e-86b5-fba6158b0a23 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.118417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782e582-9604-4287-bd3b-567c464e4237 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.123617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.124271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.124546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.124816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.132498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.134026 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-962c75c7-9ac5-4fd0-b2df-f77f0be6e6b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:31.146413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:31.147306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:31.148572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:31.152277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 254/514] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:31.152902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.154667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.154948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.155217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.159740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.160703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.161217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.161693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/54179215880d4ed291d18944a2f86fdc/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.164073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=54179215880d4ed291d18944a2f86fdc, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.167609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.173513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:31.174816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:31.248382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 261/515] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:31 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/54179215880d4ed291d18944a2f86fdc/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:31.249984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.251025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 255/516] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:31.270088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.287272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.383676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ee8d-520e-4c2a-a591-d867c3a1a327 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.422836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.423508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.423783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.424070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.424172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12408f-9623-4853-9de6-056d9735ccdc None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.443238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.444917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.463773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.464564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8d82407c-6d28-4c7e-8310-033e3355ee34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:31.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:31.465538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:31.466397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:31.466397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:31.466397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:31.466582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.466582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.466713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 256/517] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:31.466782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:31.466782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.467230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d04aa-513a-4566-9a62-12807a30747e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.467349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d04aa-513a-4566-9a62-12807a30747e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.471952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d04aa-513a-4566-9a62-12807a30747e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.473012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf5ac4-a2da-489f-b7c9-e312fd7c5b29 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.473173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 257/518] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:31.474496 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d04aa-513a-4566-9a62-12807a30747e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.493487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 262/519] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:31.494791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.495081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.495348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.518582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:31.519908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:31.550580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.564040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.587156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d04aa-513a-4566-9a62-12807a30747e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.596210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 263/520] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:31.596210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.641377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.641950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.642235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.642504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.643150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:31.643894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.645653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.663305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 258/521] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:31.664048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.693575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20847724-9c58-4f11-aaaa-962bcd817498 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.729409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.730129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.730475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.730845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.731644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=c3e348fd8e2b4203b7bc43240cb79d74) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:31.732041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.733548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.741019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 264/522] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:31 2022] GET /identity/v3/role_assignments?user.id=c3e348fd8e2b4203b7bc43240cb79d74&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:31.741892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.790339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.808205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be1619-9181-49a2-a8a5-5db076e03e01 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.855938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:31.877448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.878251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.878603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.878948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.879615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.881823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:31.895200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:31.896883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:31.897704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:31.898415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 259/523] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Oct 10 16:50:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:31.908259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:31.927876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8df47-cfa9-4f87-b270-55ce0fc19c09 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:31.967097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:31.967834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:31.968016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:31.968281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:31.968930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:31.970375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:32.029368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fe795-876d-4fe8-ab58-ea1eef99a996 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:32.064910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:32.065515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:32.065786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:32.066052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:32.066678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:32.066970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:32.068374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:32.076194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 260/524] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 10 16:50:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:32.076194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:32.092467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 265/525] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:50:31 2022] POST /identity/v3/users => generated 254 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:32.094625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:32.158952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:32.239381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87889-db47-485a-aaa6-1745bc3c0dbd None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:32.309970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:32.311095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:32.311491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:32.312664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:32.312799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] PATH_INFO: `/v3/services/network` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:32.316244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:32.316244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:32.320143 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460528b6-63ae-44bc-b132-d3bd6d4900ba None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:32.331125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:32.332234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:32.333636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:32.334377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 261/526] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:32 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:32.339753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:32.362728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:32.363294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:32.363573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:32.363847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:32.364592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:32.364843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:32.366248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:32.372661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 266/527] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 10 16:50:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:32.479244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f7416-33c5-4c0d-841f-a8f4781eee66 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:32.498382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:32.524400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:32.525113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:32.525391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:32.525668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:32.526364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] RBAC: Adding query filter params (name=network) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:32.526669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:32.528185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd07a0-7513-4a3c-bf61-6b2616896173 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:32.536538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 262/528] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:32.541808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:32.655383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97e8cd-d95c-43ef-8394-770a08c02de7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:32.692125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:32.693272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:32.693548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:32.693814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:32.694446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] RBAC: Adding query filter params (type=network) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:32.694742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:32.696118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97d721-ed14-483f-944f-48e4d2787e64 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:32.701230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 267/529] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:33.897611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19354-a127-4d00-8316-46cbfa735722 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:33.898158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19354-a127-4d00-8316-46cbfa735722 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:33.898158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f19354-a127-4d00-8316-46cbfa735722 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:33.899744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 263/530] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:33.905307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac548b-a5ad-4307-aaa8-bbfbd614f729 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:33.905735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac548b-a5ad-4307-aaa8-bbfbd614f729 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:33.905924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac548b-a5ad-4307-aaa8-bbfbd614f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:33.933827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f2ac548b-a5ad-4307-aaa8-bbfbd614f729 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:33.935025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f2ac548b-a5ad-4307-aaa8-bbfbd614f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:33.956383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:34.012104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 268/531] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:34.275953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19354-a127-4d00-8316-46cbfa735722 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:34.390414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca985da2-69b5-41ca-a374-9a2990460fc3 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.390783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca985da2-69b5-41ca-a374-9a2990460fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.390964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca985da2-69b5-41ca-a374-9a2990460fc3 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.392392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 269/532] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:34.400803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.401089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.401369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.404457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f19354-a127-4d00-8316-46cbfa735722 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:34.432782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:34.433979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:34.440352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:34.441865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.442133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.443031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.454850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:34.459747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:34.460671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:34.461525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:34.466148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 264/533] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:34.467824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:34.552173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 270/534] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:34.564378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:34.618593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6d08d-dca5-46c8-ad8b-e142d51e8af1 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:34.663573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:34.664258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.664544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.664819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.665484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:34.665789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:34.667159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:34.680266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 265/535] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:34.686083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:34.808450 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:34.827838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb385db-f180-4c03-b98b-a97dbb1ec4b1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:34.878637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:34.879283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.879562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.879867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.900410 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:34.901887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-f4aa3817-1ba7-4501-9d6d-c7fd40995324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:34.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:34.925834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:34.927294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:34.927294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:34.927294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:34.929686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 271/536] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:34.935691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:34.955692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087685-4293-4a5a-ac46-3df0de59eeae None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.956348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087685-4293-4a5a-ac46-3df0de59eeae None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.956819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087685-4293-4a5a-ac46-3df0de59eeae None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.958638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 272/537] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:34.964375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:34.964659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:34.964935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:34.985369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:34.986569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:35.076441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 273/538] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 16:50:35.076813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce17e-b298-439a-94da-c070718ee2d6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.077309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.080431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.114285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.115245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.115725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.116217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.117070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:35.117574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.119121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a70d94-e8c8-47a0-9f6c-69adead7e965 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.131202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 266/539] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:35.132784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7fe1-7814-4073-9d33-73bfa1d8c1ad None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.133364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7fe1-7814-4073-9d33-73bfa1d8c1ad None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.133830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7fe1-7814-4073-9d33-73bfa1d8c1ad None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.135498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 267/540] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:35.136434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec7fe1-7814-4073-9d33-73bfa1d8c1ad None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.169437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.217588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e8f9-da94-4f1f-bad9-ffc161d65c09 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.245589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec7fe1-7814-4073-9d33-73bfa1d8c1ad None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.264046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86def-d3a7-4b58-816f-de644c9da29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.265051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86def-d3a7-4b58-816f-de644c9da29a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.265349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86def-d3a7-4b58-816f-de644c9da29a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.265625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86def-d3a7-4b58-816f-de644c9da29a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.280577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b86def-d3a7-4b58-816f-de644c9da29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-e2b86def-d3a7-4b58-816f-de644c9da29a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:35.289732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:35.290683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:35.291521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:35.292847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.293804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.294329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.294784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.295742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 274/541] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:35.296989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.297320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.297901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.304397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.306455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.319774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:35.319957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:35.321022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:50:35.321939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:35.322462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:35.324532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 268/542] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:35.325608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.352845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.397457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 275/543] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:35.404256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.445817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1f6ca-e805-476b-b5a7-ef527ee3d6c0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.457136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.498256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.499120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.499705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.500531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.501668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:35.502078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.504534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.506694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e672869-e960-4465-a412-6aa4287a0b7f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.516265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 269/544] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:35.517290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.542182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.542785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.543157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.543432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.544194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:35.544537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.546232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.564744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 276/545] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:35.565576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.619864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.634719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd844407-70fc-4511-81f7-b2d2ee359a22 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.669315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.669931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.670206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.670503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.672566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cac47d-6ad8-4877-9b45-d428141e2e32 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.681141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:35.685865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:35.686791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:35.687619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 270/546] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:35.688879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.689820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.710767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.711391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.711662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.711928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.722338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.723932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-abf14aec-67d8-4183-a495-3df071d65685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:35.739237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:35.740977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:35.742097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:35.742097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:35.742097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:35.742097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 277/547] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:35.742097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5b816-06e4-43db-99e7-0f19b6507a60 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.742097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5b816-06e4-43db-99e7-0f19b6507a60 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.742440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5b816-06e4-43db-99e7-0f19b6507a60 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.743559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 278/548] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:35.744265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5b816-06e4-43db-99e7-0f19b6507a60 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.796890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd33e-d02e-46a7-991c-08a497ac2732 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.830629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.831236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.831514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.831784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.846656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.863854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5b816-06e4-43db-99e7-0f19b6507a60 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:35.897523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:35.898065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:35.898344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:35.898614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:35.899276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:35.899580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.900999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.902046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:35.903477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:35.908833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 279/549] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:35.909585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:35.938564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:35.998168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 271/550] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 16:50:35.999003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.011791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ad71-d60c-4480-9c91-923b14a94868 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.042687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.047258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.047866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.048169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.054086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.054086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.054086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.094212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b8254-975e-40f4-9395-97fda4dde5f5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.126849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.127512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.127758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.128101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.128679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:36.128974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.130330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe9ec7-74b2-4ac9-b051-5e120bbfaf66 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.141470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 272/551] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:36.142565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.142842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.143111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.154922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 280/552] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:50:35 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:36.155517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.162571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:36.163796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:36.260081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 273/553] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 16:50:36.260081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.264177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.279489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd7fa-9e8e-4c76-8425-10cb777f8603 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.299152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.321669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.322319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.322602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.323003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.335717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.337194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:36.353260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:36.354274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:36.355147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:36.355147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:36.355147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:36.359032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 281/554] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:36.359909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.391802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688bbab-d92f-439f-86fd-70d6203d7fcb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.441627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.442914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.443204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.443486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.456708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.459354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.474183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a74f-1d7b-48fc-b549-c0ad26f4313d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:36.479210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:36.480170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:36.481071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:36.481071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:36.481071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:36.481071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 274/555] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:36.482082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.510677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.511458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.511842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.512209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.520187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.522355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.525563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:36.534719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:36.535910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:36.537101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:36.542653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 282/556] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:36.542653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.587116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.602816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e2d0-d513-4bc0-8620-6878221a6151 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.642607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.643414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.643693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.643964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.644640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:36.644944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.646422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.649148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13efb2-6547-4871-88a1-1242992995c7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.657836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 275/557] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:36.658635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.691951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.693159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.693508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.693849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.694939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:36.694939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.696417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.715863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 283/558] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:36.715863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.772257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaad95-a8c1-4514-ae7e-a31838009a36 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.807329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.807937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.808244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.808516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.809162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.810524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.818076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.856880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2f259-b662-4d6f-b062-cf2d3e53525a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:36.885714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 276/559] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:50:36 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:36.886385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:36.909609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:36.910551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:36.911045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:36.911599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:36.911939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:36.912742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:36.913052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:36.914835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:36.941264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 284/560] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:36.942260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.006025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d3edf-120d-4b20-90f6-39248943c222 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.044941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.046009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.046595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.047091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.057247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.059131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:37.073278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:37.074277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:37.075227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:37.076782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c135659-316f-4055-9978-fb2924df6409 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.081653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 277/561] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:37.081653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.095990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:37.125013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.125660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.125943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.126216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.137239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.138772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:37.156623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:37.157756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:37.158755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 285/562] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:37.164672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:37.164872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.217156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7ded8-c5dd-488b-8ba8-6747971b59eb None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.254913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.255392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.255672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.255942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.267697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.269229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.279424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02333a68-c6b4-42b3-bef1-1aab601f73cc None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:37.284953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:37.286639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:37.287549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:37.287549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:37.287549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:37.287549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 278/563] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:37.287549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.319240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.320217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.320643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.321105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.321977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:37.322428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.324072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.347024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 286/564] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:37.348148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.380013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:37.402612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2fdf-1134-4b7a-8b59-a01f3754b6f7 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.420063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:37.442970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.443583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.443857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.444168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.444960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:37.445262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.446848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.454618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0b8b0-84af-4a23-8fb1-7f834121902e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.460120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 279/565] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:37.460120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.503388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.503915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.504221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.504542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.505380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:37.505687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.507626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.522322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 287/566] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:37.523235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.609344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624619f-cb9d-40ad-b059-8ba54f9f4af4 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.631447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:37.644458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4d697-f094-430f-95f2-b7e8f1069416 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.646950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.648237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.648237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.648237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.659208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.661650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:37.677425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:37.678587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:37.679742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:37.679742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:37.679742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:37.681732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 280/567] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:37.682637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.698367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.699045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.699331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.699606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.700821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=54179215880d4ed291d18944a2f86fdc) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:37.701135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.702968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.716185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 288/568] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/role_assignments?user.id=54179215880d4ed291d18944a2f86fdc&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:37.717293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.725511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:37.851289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f33c9-9043-407b-87b7-06bc5ec26f61 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.888937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.889574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.889845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.890107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.898031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.899480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:37.911622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:37.912869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:37.913942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 281/569] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:37.918742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:37.941714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8aed76-34b9-4c22-9c53-733933c0dde2 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:37.986741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:37.987791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:37.988356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:37.988837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:37.989715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:37.990279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:37.991992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:38.006152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 289/570] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:38.011866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:38.032167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:38.037495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:38.049382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6e7ce-8f37-4b86-92b2-813e35005797 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:38.084518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:38.085135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.085410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.085678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.086374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:38.086687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:38.088613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:38.102890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 282/571] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:38.107664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:38.126719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237db6e-7597-4698-8437-bb8ac65ad2dd None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:38.165444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:38.166654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.167169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.167643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.175643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:38.178109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:38.191147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:38.192710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:38.193641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:38.195891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 290/572] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:38.201909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:38.212610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4c312-e7d5-4f37-883b-4fccc7841eef None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:38.247207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:38.248130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.248336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.248573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c3e348fd8e2b4203b7bc43240cb79d74/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.250500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=c3e348fd8e2b4203b7bc43240cb79d74, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:38.251898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:38.289524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:38.315362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83ac7-5c4b-4dee-87c4-3b7f9ea4bafe None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:38.323722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 283/573] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:38 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c3e348fd8e2b4203b7bc43240cb79d74/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:50:38.353492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:38.354456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.354944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.355437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.356453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:38.356995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:38.358641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49bb30-7203-4926-aa0a-20e6776e7a6d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:38.373587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 291/574] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:38.380390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:38.392981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:38.524135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7fba-c50c-4f87-a1ee-5dda2813f14b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:38.555921 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24cd78-ab59-4c8f-8d1e-ceaebe965382 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.556610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24cd78-ab59-4c8f-8d1e-ceaebe965382 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.557088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24cd78-ab59-4c8f-8d1e-ceaebe965382 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.558911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 292/575] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:38.564128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.564665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.565170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.572387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960683ce-918a-4137-ad62-fdef8c69f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:38.573075 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960683ce-918a-4137-ad62-fdef8c69f850 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:38.573357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960683ce-918a-4137-ad62-fdef8c69f850 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:38.573635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960683ce-918a-4137-ad62-fdef8c69f850 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c5b6addb805b4d0a89ee2cd9ac3de27a/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:38.587172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:38.588844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:38.606167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960683ce-918a-4137-ad62-fdef8c69f850 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=c5b6addb805b4d0a89ee2cd9ac3de27a, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:38.607537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960683ce-918a-4137-ad62-fdef8c69f850 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:38.670175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 293/576] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:38.731915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 284/577] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:38 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c5b6addb805b4d0a89ee2cd9ac3de27a/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:39.087973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:39.166684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:39.210471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282c60d-7207-4db3-ac85-d8d8d67b3ceb None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:39.215255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859d904-8965-4291-bc93-f10734795bc0 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:39.215709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859d904-8965-4291-bc93-f10734795bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:39.215788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859d904-8965-4291-bc93-f10734795bc0 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:39.217411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 285/578] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:39.222437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:39.222728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:39.222997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:39.248424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:39.249622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:39.268705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:39.269338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:39.269612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:39.269876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:39.271166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:39.272578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:39.287311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:39.288407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:39.289655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 294/579] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Oct 10 16:50:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:39.294286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:39.356764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:39.391800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 286/580] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:39.441337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752d16-4a6a-42d3-a7e8-2c19277bf014 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:39.475107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:39.485863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:39.486481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:39.507181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:39.507181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:39.507181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:39.507181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:39.507181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:39.507181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 295/581] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 10 16:50:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:39.507608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:39.644523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df433d-d4d5-41e6-bce7-a72c0a5e012a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:39.678015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:39.678621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:39.678894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:39.679161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:39.679782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:39.680104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:39.681711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:39.687767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 287/582] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 10 16:50:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:39.693873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:39.820905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13deef7-fc3f-4ba6-955f-3abe2738e9bb None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:39.857119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:39.864803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:39.865427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:39.865709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:39.865985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:39.866760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] RBAC: Adding query filter params (interface=public, service_id=3cd7e1d56bb54e149511fdc6c03a2ec7, region_id=RegionOne) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:39.867068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:39.868546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:39.876739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 296/583] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:50:39 2022] GET /identity/v3/endpoints?service_id=3cd7e1d56bb54e149511fdc6c03a2ec7&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:39.882827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:39.948612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:39.964391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:39.987491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72e18-3854-472e-8124-76f07848e27a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:40.034042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fa9e9-13ed-4d83-8204-4f133d66174e None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:40.057499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:40.058464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.058940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.059431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.060314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:40.061886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:40.073613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:40.074249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.074532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.074808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] PATH_INFO: `/v3/services/network` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.075513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:40.075943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 297/584] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:50:39 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:40.078675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:40.086656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:40.087759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:40.088688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 288/585] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:39 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:40.093698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:40.246053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492e3f-0f14-41ab-846b-5dfd9ec85b13 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:40.278134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:40.296501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:40.297139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.297412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.297678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.298322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] RBAC: Adding query filter params (name=network) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:40.298622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:40.300002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:40.308847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 298/586] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:40.314129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:40.436072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a5f6-4703-4c5e-9753-1481be354dbe None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:40.454913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:40.478690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:40.479311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.479588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.479858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.480531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] RBAC: Adding query filter params (type=network) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:40.480835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:40.482203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af4641-3efc-4d99-8132-7eeee79b8e8e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:40.488127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 289/587] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:40.493589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:40.572587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b3dab-b27f-4a0a-bfc0-562a85de3cb7 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.573192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b3dab-b27f-4a0a-bfc0-562a85de3cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.573652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b3dab-b27f-4a0a-bfc0-562a85de3cb7 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.575210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 290/588] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:40.593273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.593549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.593821 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.624751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:40.628292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:40.642719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d25cc-4a25-4055-9304-992403e7a130 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:40.709598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:40.710223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.710501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.716863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.716863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] RBAC: Adding query filter params (interface=public, service_id=cf735c82f1354e6bb2054392d4fb9453, region_id=RegionOne) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:40.716863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:40.716863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:40.721044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 299/589] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:50:40 2022] GET /identity/v3/endpoints?service_id=cf735c82f1354e6bb2054392d4fb9453&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:40.724612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:40.754105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 291/590] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:40.824482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:40.906781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0ed8-312e-4353-9d49-df9d4183f817 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:40.952803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd61a8-d1f6-4f5c-82d4-79b84bf97c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:40.953500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd61a8-d1f6-4f5c-82d4-79b84bf97c9b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:40.953784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd61a8-d1f6-4f5c-82d4-79b84bf97c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:40.954111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd61a8-d1f6-4f5c-82d4-79b84bf97c9b None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:40.954717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd61a8-d1f6-4f5c-82d4-79b84bf97c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:40.956150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd61a8-d1f6-4f5c-82d4-79b84bf97c9b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:40.964296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 300/591] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:50:40 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:41.102911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:41.226696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bde7b-8c92-4ab9-91bb-0dc1b8e6d1f7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:41.246861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:41.252643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86a87-8a43-4d9b-9d3c-1b0635fe85cb None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.253173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86a87-8a43-4d9b-9d3c-1b0635fe85cb None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.253253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86a87-8a43-4d9b-9d3c-1b0635fe85cb None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.255577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 301/592] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:41.261346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd684c01-46e6-47bd-9c4f-98b626de0e91 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.261633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd684c01-46e6-47bd-9c4f-98b626de0e91 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.261899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd684c01-46e6-47bd-9c4f-98b626de0e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.280082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:41.281108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.281612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.282079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.282868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:41.284544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:41.294719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:41.295726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:41.296656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:41.299468 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 292/593] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 10 16:50:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:41.304069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-cd684c01-46e6-47bd-9c4f-98b626de0e91 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:41.304345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cd684c01-46e6-47bd-9c4f-98b626de0e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:41.308080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:41.394132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 302/594] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:41.450150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151a474-7223-4f5d-83d2-5d30f640bea8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:41.480337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69130f-98ee-4790-a88e-0df2cca29eea None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.481210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69130f-98ee-4790-a88e-0df2cca29eea None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.481684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69130f-98ee-4790-a88e-0df2cca29eea None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.484350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 303/595] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:41.490301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.490824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.491310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.491575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:41.492229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.492510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.492782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.493415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:41.493723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:41.495065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:41.510352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 293/596] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:41.512764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:41.514252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:41.515560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:41.564741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:41.569760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:41.619763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 304/597] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:41.626163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80056e0-7ffd-4574-9f31-ae11db4ff11b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:41.697689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:41.698309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.698585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.698853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.699483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:41.699786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:41.701232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:41.711896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 294/598] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:41.760791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:41.873762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61126c8c-ebf5-4269-b0be-935f92f0db83 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:41.905984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:41.908768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:41.909490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:41.909772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:41.910041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:41.922644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:41.927795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:41.928773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:41.930053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 305/599] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:41.935249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:41.975361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.013034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d82dc-c282-4ea4-84da-9d2e8b79a6ec None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.026871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.047519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc525861-3ada-415b-9c99-72bd90c9cf69 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.072241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.072896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.073178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.073452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.088095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:42.093887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:42.095034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:42.096063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 295/600] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:50:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:42.100630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.101515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.101623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.102186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.102654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.103576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:42.104127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.105718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.115260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 306/601] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:42.121094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.241156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.247977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e89ee-c432-40f8-805d-c2bd53e4f9a1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.286581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.287975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.288471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.289293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.294285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f033ee-2193-4ffa-af30-e5d436224e39 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.301916 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.303579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:42.320227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:42.321205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:42.322072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:42.322072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:42.322072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:42.328254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 307/602] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:42.330244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.405662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.406206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.406503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.406780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.408128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:42.408128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.409230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.440258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 296/603] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:42.446170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.448835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708427ee-e7a4-4a03-8db5-6480c48236f6 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.492190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.493038 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.493483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.493904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.494691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:42.495242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.496860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.510248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 308/604] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:42.515910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.519556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.544398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.622351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926763c-77b1-4703-b0ab-750006f6c922 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.651758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f372c-8fbd-49d2-882d-782c408a1003 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.658114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.659000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.659463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.659915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.668678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.670406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-5e9be43b-d4a0-471c-a736-1e3e0b2245ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:42.682942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:42.683980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 16:50:42.685013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:42.687843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 309/605] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:42.694167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff14f8-a481-4160-b17f-d814221e790e None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.694451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff14f8-a481-4160-b17f-d814221e790e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.694724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff14f8-a481-4160-b17f-d814221e790e None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.696151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 310/606] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:42.700320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff14f8-a481-4160-b17f-d814221e790e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.714714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.714714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.714714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.714714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.715068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.715068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-f5a21088-1264-41d0-8daf-c48cc49d94ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:42.735730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:42.736755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:42.737634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:42.737634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:42.737634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:42.737634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 297/607] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:42.738593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.738920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.739192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.762870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:42.764676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:42.782063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.824758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff14f8-a481-4160-b17f-d814221e790e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:42.846157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 298/608] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:42.847478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:42.868369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:42.874881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:42.876453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:42.876873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:42.877464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:42.879598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:42.880819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:42.883498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:42.922838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 311/609] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:42.923894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.047811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ebad-5615-448b-a133-09c2135a1ac4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.079961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.092639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.093274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.093550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.093817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.101717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75841b81-acfa-45be-81ee-1cfd3103622f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.156223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.156858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.157134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.157400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.158402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:43.158710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.160438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.175094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 299/610] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:43.176091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.187389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.189297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.256431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.299886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2674d-c193-4b4b-851c-a622edb08ce5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.306432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 312/611] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:50:43.307984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.340881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.347117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.347976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.348705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.349976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.354729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.354729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:43.364673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:43.365895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 16:50:43.366930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:43.369743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 300/612] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:43.370688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.438576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a283a28-28bc-4434-b44f-d334e66cedc7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.484645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0e1e4-825c-4c5c-8b1f-87e88b0e0831 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.488357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.489715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.490142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.490558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.499890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.502061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:43.514558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:43.515522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 313/613] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:43.516500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.526160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.534956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.540407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.540407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.540407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.551081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.553525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:43.578173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:43.579326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:43.580440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:43.580440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:43.580440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:43.582281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 301/614] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:43.583632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.620145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.635835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5470c5-8280-4dcd-b02c-26926fe6bc7d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.671863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.672730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.673011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.673287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.673938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:43.674245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.675708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.684116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 314/615] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:43.685550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.714812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d6de2-03aa-48c2-94a5-1519ff95138c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.752365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.753526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.754078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.754599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.755575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:43.756114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.757845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.777470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 302/616] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:43.778907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.814752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf176a5-fe6a-489a-aba0-f71e64dd70b1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.826489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.849206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.849932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.850205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.850470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.851102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:43.851398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.852828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.872786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 315/617] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:43.874118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.887093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:43.899745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82388-00ad-4d39-8b9a-93c8ce089f39 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:43.940861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:43.941781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.942104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.942371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.943147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:43.943451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:43.945530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4b418-1fee-437d-9d94-968efcac2577 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:43.951342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 303/618] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:43.952618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792a7ab-5dcc-40ee-a3a0-9f50f0f9331f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:43.952905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792a7ab-5dcc-40ee-a3a0-9f50f0f9331f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:43.953180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792a7ab-5dcc-40ee-a3a0-9f50f0f9331f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:43.954379 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 304/619] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:43.955073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792a7ab-5dcc-40ee-a3a0-9f50f0f9331f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:43.987482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafc5ec-e530-46e6-a699-018d7cc4cbcc None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.022068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bd155-0f47-44a4-a3cb-3dd003931877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.022678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bd155-0f47-44a4-a3cb-3dd003931877 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.022947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bd155-0f47-44a4-a3cb-3dd003931877 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.023223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bd155-0f47-44a4-a3cb-3dd003931877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.060193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792a7ab-5dcc-40ee-a3a0-9f50f0f9331f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.062506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.092462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bd155-0f47-44a4-a3cb-3dd003931877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.093834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bd155-0f47-44a4-a3cb-3dd003931877 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.096460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.097096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.097375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.097650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.104927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.106051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.106342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:44.117689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:44.122942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 305/620] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:44.124181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.202904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 316/621] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:50:44.204212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.204498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.204772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.225824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:44.227235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:44.233389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660f203-0b99-4f62-901d-81a00a85320b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.267671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.268320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.268595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.268867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.269538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.271485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.297847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 317/622] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:44.303732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.341824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.386317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 306/623] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:50:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:44.387433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.407859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d275fc-15ee-44f3-a4a7-0cf39b8a8863 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.428011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.466087 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.467188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.467715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.468249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.469176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:44.469731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.471508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76731066-f8e9-4222-9b8c-abac3195ee16 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.486810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 318/624] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:44.495179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bcd88-e6ff-4917-a097-22f803f5fb6c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.495179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bcd88-e6ff-4917-a097-22f803f5fb6c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.495179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bcd88-e6ff-4917-a097-22f803f5fb6c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.495179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 319/625] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:44.495179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bcd88-e6ff-4917-a097-22f803f5fb6c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.544673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3cc2f-4ec4-420b-a3d0-8864a93c6021 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.584787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.585424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.585705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.585978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.598328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.599807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.612803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-df5d4481-320e-4b06-8cd7-71f468262346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:44.616392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:44.617523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:44.618531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:44.618531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:44.618531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:44.618531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 307/626] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:44.618775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.619216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.619459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.622394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bcd88-e6ff-4917-a097-22f803f5fb6c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.638189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:44.639369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:44.657858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.658568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.658846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.659114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.659860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=c3e348fd8e2b4203b7bc43240cb79d74) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:44.660195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.661676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.667540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 320/627] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/role_assignments?user.id=c3e348fd8e2b4203b7bc43240cb79d74&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:44.668162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.728617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 308/628] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:44.729727 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.732263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.868721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001df570-9b3f-43c0-9cef-3f43d1b18843 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.910713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.912265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.912736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.913468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] PATH_INFO: `/v3/services/network` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.914489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.916778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.918184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef0e6-104d-4f3f-a3d2-3a52512f715f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:44.928714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:44.929733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 16:50:44.930577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:44.933934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 321/629] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:44.940796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:44.967506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:44.968266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:44.968608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:44.969235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:44.969975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:44.970344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:44.971791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:44.980468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:44.987940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 309/630] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:44.993952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:45.072417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:45.074004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0faefe-4d77-48c2-814f-4ab69c6031eb None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.109084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.109697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.109976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.110244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.110869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:45.111174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.112248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b60fae-9cc3-4d31-9fea-c4e4e9744c2d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.112673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.118232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 322/631] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:45.119019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:45.149758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.150397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.150678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.150954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.158332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.159891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:45.172817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:45.173760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 310/632] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:45.175371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:45.227621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:45.243051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f3b2a-76e7-4e4f-b465-e4d7120066a6 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.289446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.290840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.291372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.291847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.293312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.295281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.307125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4581-41aa-449c-9254-1701dfaf0b5b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.331515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:45.343680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.344330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.344611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.344880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.345520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] RBAC: Adding query filter params (type=network) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:45.345915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.347300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.354499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 311/633] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:45.355502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:45.362566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 323/634] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:50:45 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:45.364852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:45.499536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834170ff-97eb-44f8-b810-8543d70f0d33 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.544336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.545396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.546363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.546849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.547776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:45.548317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.549982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.568178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 312/635] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:45.570916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:45.585195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c373-945c-46e5-9669-8e3d542eb441 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.617625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:45.621840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196cacd-b8a3-4b86-bb0a-abe01cbef7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.622458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196cacd-b8a3-4b86-bb0a-abe01cbef7dc None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.622738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196cacd-b8a3-4b86-bb0a-abe01cbef7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.623028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196cacd-b8a3-4b86-bb0a-abe01cbef7dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.623691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196cacd-b8a3-4b86-bb0a-abe01cbef7dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.625171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196cacd-b8a3-4b86-bb0a-abe01cbef7dc None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.684111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8c4fa-aa60-44c6-922f-d800592d0a4c None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:45.727460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:45.728480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:45.729402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:45.731086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:45.732134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=c5b6addb805b4d0a89ee2cd9ac3de27a) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:45.732635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 324/636] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:50:45 2022] POST /identity/v3/endpoints => generated 337 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:45.733180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:45.735064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08988352-bbc9-40da-9172-4d68f4f6098e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:45.747068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 313/637] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:45 2022] GET /identity/v3/role_assignments?user.id=c5b6addb805b4d0a89ee2cd9ac3de27a&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:46.740965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeeb8b-186e-4cfe-af2a-7e44431ad36c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:46.741486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeeb8b-186e-4cfe-af2a-7e44431ad36c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:46.741486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeeb8b-186e-4cfe-af2a-7e44431ad36c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:46.743093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 325/638] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:46.748540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e64d2-b86f-4864-a29e-de34ac0ed4d1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:46.748969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e64d2-b86f-4864-a29e-de34ac0ed4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:46.749154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e64d2-b86f-4864-a29e-de34ac0ed4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:46.767874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b43e64d2-b86f-4864-a29e-de34ac0ed4d1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:46.769095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b43e64d2-b86f-4864-a29e-de34ac0ed4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:46.850808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 314/639] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:46 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:47.116768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeeeb8b-186e-4cfe-af2a-7e44431ad36c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:47.121705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:47.123661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25c6ab-c318-4221-8b5d-cf1809e6c554 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.123956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25c6ab-c318-4221-8b5d-cf1809e6c554 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.124273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25c6ab-c318-4221-8b5d-cf1809e6c554 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.125872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 315/640] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:47.131383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.131666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.131939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.152416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:47.153608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:47.184393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:47.226442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 316/641] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:47 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:47.240743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeeeb8b-186e-4cfe-af2a-7e44431ad36c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:47.275666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:47.276531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.276858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.277135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.277711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:47.279063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:47.287936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:47.288934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:47.290863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 326/642] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:47.295273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:47.400864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab40ff-1dd1-41fe-aba8-9b8cd31f42d2 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:47.435755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:47.436679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.437132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.437576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.438384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:47.438867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:47.440411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:47.446046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 317/643] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:47.451834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:47.519801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:47.522174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:47.557161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:47.562000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534d02-6a0a-4541-8b5b-9d6250fef218 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:47.600381 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:47.601387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.601911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.602407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.603314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:47.603868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:47.605454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:47.615070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 327/644] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:47 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:47.653526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddd38e-2b1c-4f6f-8f0b-c629eea84662 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:47.696369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:47.697388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.698073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.698548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.699369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:47.701205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-2dcc0c56-ce07-4492-a159-4f8d63324928 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:47.711621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:47.719239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:47.720109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 318/645] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:47.722792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:47.784369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1b35-c274-40a4-8b5c-2f15319d6908 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.784904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1b35-c274-40a4-8b5c-2f15319d6908 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.784904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1b35-c274-40a4-8b5c-2f15319d6908 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.791337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 319/646] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:47.793193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114edea-e862-4a2a-b4cc-ddc38e0e0b88 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.793487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114edea-e862-4a2a-b4cc-ddc38e0e0b88 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.793764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114edea-e862-4a2a-b4cc-ddc38e0e0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.817225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d114edea-e862-4a2a-b4cc-ddc38e0e0b88 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:47.819102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d114edea-e862-4a2a-b4cc-ddc38e0e0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:47.850348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60b63f-82e3-4a32-82ef-3761aca0e6ac None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:47.888408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:47.897120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:47.897734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:47.898017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:47.898296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:47.899364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:47.899788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:47.901482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:47.912721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 328/647] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:47.912867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 320/648] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:47 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:47.913806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.019645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80373-aa84-457e-bb6c-cec3fabdf5bd None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.063855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.064618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.064892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.065158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.065787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:48.066082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:48.067487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6de723-2e60-4961-944d-d2bac2b653e7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:48.072407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 329/649] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:48.147558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634a7c2-6c34-4437-9bf7-a80667698930 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.147888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634a7c2-6c34-4437-9bf7-a80667698930 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.148145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634a7c2-6c34-4437-9bf7-a80667698930 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.149821 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 321/650] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:48.155351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.155658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.155917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.176223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:48.177446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:48.205849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634a7c2-6c34-4437-9bf7-a80667698930 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.246080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:48.253145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:48.266462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 330/651] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:48 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:48.315845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634a7c2-6c34-4437-9bf7-a80667698930 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.352055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.352691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.353023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.353294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.353874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:48.355273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:48.364337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:48.365271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:48.366554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 322/652] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:48.371534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.472589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b430e-703d-46ba-b16e-50adc10378ca None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.506739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.507348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.507621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.507888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.508541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:48.508838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:48.510193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:48.515151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 331/653] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:48.521244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.546347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.617672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:48.630259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:48.643036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fb338-56a6-4c07-b34f-c9f52957d4db None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.668281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f5580-d45e-48e9-8871-935d99a20aac None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.686620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.687694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.688226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.689007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.689903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:48.690412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:48.693111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:48.700141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 323/654] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:48.706412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.714023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.714661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.714940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.715211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.728327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:48.737596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:48.746528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 332/655] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:48.747472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.834150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114c511-afe4-4756-aac4-605b90026385 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.887041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21a505-9d14-4412-b60e-8efc686fbdfe None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:48.887867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.888931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.889380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.890754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.891676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] RBAC: Adding query filter params (interface=public, service_id=a1a5a69b17e1498a91823eee7c8760ad, region_id=RegionOne) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:48.892164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:48.893689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:48.903571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:48.905186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 324/656] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/endpoints?service_id=a1a5a69b17e1498a91823eee7c8760ad&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:48.910767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:48.927997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:48.934088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:48.935157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:48.935614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:48.936174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:48.936855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:48.937163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:48.938998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:48.955375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 333/657] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:48.959817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:49.024049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35697ec4-e28e-4957-81d3-7973962cd807 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.063193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:49.064237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.064715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.065210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.066008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:49.066180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396bf17-2354-4245-9d6d-d452e0eec1c5 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.067808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:49.080249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 325/658] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/services => generated 1029 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 16:50:49.104623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:49.105278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.105557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.105833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.119028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:49.120583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:49.136802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:49.137797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:49.138659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:49.138659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:49.138659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:49.138659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 334/659] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:49.145284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:49.233048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:49.258262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8c2a3-e224-4947-9802-982893d545e6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.294136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:49.294664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.294937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.295204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.295839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:49.296856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:49.297502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:49.310661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 326/660] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:49.316168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:49.346823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:49.421782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a296c-cd4e-482a-b5a7-2032d9593a70 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.455764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:49.456454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.456730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.456999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.465436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:49.466866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:49.478763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:49.479919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:49.481110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 335/661] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:50:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:49.485876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:49.588233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f15bf4-0993-4de5-afd9-8602192b2114 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.622765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:49.623372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.623646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.623915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.624666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:49.625022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:49.626607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:49.629128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:49.644661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 327/662] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 10 16:50:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:49.651011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:49.745072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:49.758062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caa9f4-3e8c-4437-b42d-c2c4d1aba2a3 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.797736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:49.798417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.798690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.798987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.811034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:49.812621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-6079f99b-73dc-4bd3-9e53-6299745cc65b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:49.827512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:49.828699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:49.829802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:49.829802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:49.829802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:49.829802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 336/663] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:49.835114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:49.888809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c4da2-732a-4e67-8c3c-b30666831457 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.889325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c4da2-732a-4e67-8c3c-b30666831457 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.889325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c4da2-732a-4e67-8c3c-b30666831457 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.890943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 337/664] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:49.896577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:49.896860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:49.897135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:49.919558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:49.921129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:49.963057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93315b-05b5-4d8a-8b56-4f79765b6201 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:49.993860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 338/665] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:49 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:50.000097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:50.000977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.001562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.002045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.002889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:50.003395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:50.005574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:50.017993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 328/666] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:50.025444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:50.106557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:50.133305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ddbc-bd40-4db0-bc75-b4220d5eeaac None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:50.169566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:50.170554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.171052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.171530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.178625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:50.180314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:50.192223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:50.193136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:50.194050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:50.196364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 339/667] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:50.203420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:50.262356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:50.266998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:50.318116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b826183-f344-4180-b31e-f8e77b3f3fd2 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:50.357017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:50.358050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.358328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.358594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.359617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:50.359930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:50.361951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01973-32d8-4a6b-bda4-5d842dcbccfa None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:50.376063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 329/668] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:50.377347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b413-3a60-4f03-93f9-fc0778b3e74c None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.377635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b413-3a60-4f03-93f9-fc0778b3e74c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.377906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b413-3a60-4f03-93f9-fc0778b3e74c None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.379270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 330/669] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:50.383991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68b413-3a60-4f03-93f9-fc0778b3e74c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:50.392083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9997673-d0d9-468f-b30b-e529aa9c85e0 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:50.422853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:50.428540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62553f39-6f2b-4285-852c-891b4a34d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:50.429171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62553f39-6f2b-4285-852c-891b4a34d0f8 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.429446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62553f39-6f2b-4285-852c-891b4a34d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.429716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62553f39-6f2b-4285-852c-891b4a34d0f8 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.430398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62553f39-6f2b-4285-852c-891b4a34d0f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:50.431961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62553f39-6f2b-4285-852c-891b4a34d0f8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:50.486375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68b413-3a60-4f03-93f9-fc0778b3e74c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:50.499962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:50.519553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:50.520854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.521131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.521398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c5b6addb805b4d0a89ee2cd9ac3de27a/roles/49f33a0fd78d4b95aef3f4a074616709` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.523545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 340/670] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:50:50 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:50:50.524628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf28a-cdf0-4a82-8b4f-8d1a244e1dd3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:50.524913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf28a-cdf0-4a82-8b4f-8d1a244e1dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:50.525190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf28a-cdf0-4a82-8b4f-8d1a244e1dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:50.528370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=c5b6addb805b4d0a89ee2cd9ac3de27a, role_id=49f33a0fd78d4b95aef3f4a074616709)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:50.530461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:50.546196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e9baf28a-cdf0-4a82-8b4f-8d1a244e1dd3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:50.547428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e9baf28a-cdf0-4a82-8b4f-8d1a244e1dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:50.618139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 331/671] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:50:50 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/c5b6addb805b4d0a89ee2cd9ac3de27a/roles/49f33a0fd78d4b95aef3f4a074616709 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:50:50.635931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 341/672] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:50.911996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:51.015355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded43119-1ad7-45ab-a8ba-f52e8c2b3968 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:51.049292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60b8f-71fc-40da-81cf-79c647a389a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:51.049902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60b8f-71fc-40da-81cf-79c647a389a2 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:51.050180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60b8f-71fc-40da-81cf-79c647a389a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:51.050448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60b8f-71fc-40da-81cf-79c647a389a2 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:51.051628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60b8f-71fc-40da-81cf-79c647a389a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:51.053093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60b8f-71fc-40da-81cf-79c647a389a2 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:51.120347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 332/673] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:50:50 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:51.438191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba8e20-22c5-4688-9f90-df1ba65fd256 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:51.438709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba8e20-22c5-4688-9f90-df1ba65fd256 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:51.438709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba8e20-22c5-4688-9f90-df1ba65fd256 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:51.440380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 342/674] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:51.445692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:51.445986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:51.446265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:51.464673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:51.465877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:51.494242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:51.544707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 333/675] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:51.811792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba8e20-22c5-4688-9f90-df1ba65fd256 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:51.916107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba8e20-22c5-4688-9f90-df1ba65fd256 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:51.932605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:51.950733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:51.951368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:51.951644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:51.951914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:51.952525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:51.953895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:51.963273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:51.964212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:51.965092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 343/676] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 10 16:50:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:51.970292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:52.077330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e70366-801f-4d5a-9d4f-e25c4b69444e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:52.111328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:52.112014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.112320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.112591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.113216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:52.113518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:52.114902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:52.119915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 334/677] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:52.125236 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:52.232324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ee682-4b1c-4572-8ba7-c5c489d72b8c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:52.266794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:52.267752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.268271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.268766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.269586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:52.270651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:52.272247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da517c-c7a2-4d73-b3bc-f24ccf3dfb27 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:52.277795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 344/678] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:52.283248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:52.320404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:52.386909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9097f-4160-404a-a014-93ea90478cbe None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:52.422288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952f0c0-f1f2-4e0a-9619-00868944cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:52.422898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952f0c0-f1f2-4e0a-9619-00868944cf1b None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.423176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952f0c0-f1f2-4e0a-9619-00868944cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.423449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952f0c0-f1f2-4e0a-9619-00868944cf1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.424146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952f0c0-f1f2-4e0a-9619-00868944cf1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:52.425497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952f0c0-f1f2-4e0a-9619-00868944cf1b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:52.508666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 335/679] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:50:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:52.835819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0950e0e-9150-467d-aa8e-8ec9eb5f0c91 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.836492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0950e0e-9150-467d-aa8e-8ec9eb5f0c91 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.836492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0950e0e-9150-467d-aa8e-8ec9eb5f0c91 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.838292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 345/680] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:52.843390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.843677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.843948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.860556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d48362-c79e-44ab-a2cb-a27bfa44ea5e None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.860859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d48362-c79e-44ab-a2cb-a27bfa44ea5e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.861093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d48362-c79e-44ab-a2cb-a27bfa44ea5e None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.862677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 346/681] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:52.869786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:52.869936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:52.869936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:52.869936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:52.870148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:52.891810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:52.893520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:52.946500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 336/682] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:52.960391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:52.975025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 347/683] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:53.217056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.226087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:53.244865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.326671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27f29e-4ade-45c0-92c3-aef519470e4f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.362795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.363437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.363744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.364013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.374365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:50:53.377195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a93be-28eb-4061-99d3-ad37c8d49e66 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:50:53.379253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:53.380230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:53.381130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 337/684] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:53.386272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.425374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.426060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.426346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.426616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.427216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:53.428662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-7b4e2d85-9d57-413c-bd91-ce9483290a32 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:53.438700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:53.439832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:53.441094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 348/685] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:53.442448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72cac6-df5a-4276-a087-7bffbabbf0fb None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.442448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72cac6-df5a-4276-a087-7bffbabbf0fb None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.442448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72cac6-df5a-4276-a087-7bffbabbf0fb None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.444077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 349/686] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:53.446056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72cac6-df5a-4276-a087-7bffbabbf0fb None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.468508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:53.491885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:53.502928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e401b-6f3e-4f83-ab54-67215b4325c9 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.550601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.551547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.552054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.552530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.553377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:53.553857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:53.555404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34f6ea-4737-4a6b-bf68-521e041fab33 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:53.564443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72cac6-df5a-4276-a087-7bffbabbf0fb None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.569224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 338/687] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:53.570287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.570573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.570848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.589527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:53.590723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:53.600544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.601147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.601727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.601999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.602766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:53.603067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:53.604436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:53.609420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 350/688] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:53.610008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.676050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 339/689] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:53.676050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.712093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cd78-a4b1-4f67-96e2-f40811eb17f8 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.713849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:53.746786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.746984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:53.747523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.747803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.748109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.760352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:53.761813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:53.777542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:53.778863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:53.779738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:53.779738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:53.779738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:53.779738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 351/690] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:53.784237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.803915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45ab1a-7317-4197-a7f5-ec78d41a9523 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.850320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.851449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.851905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.852397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.853220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:53.853939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:53.856345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:53.862878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 340/691] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:53.872563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.907262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee2810-5325-4a21-940e-4edf8cfc2dad None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:53.945443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:53.946306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:53.946691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:53.947337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:53.947901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:53.948332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:53.949850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:53.967712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 352/692] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:53.968834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:53.984430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:53.991014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb1589-a5f8-42e3-9578-cbe5ded985fb None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.013726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.026014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.026907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.027194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.027471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.028367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] RBAC: Adding query filter params (interface=public, service_id=8025d521668943bf96305daab8347b42, region_id=RegionOne) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:54.029002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.030802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.040162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 341/693] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/endpoints?service_id=8025d521668943bf96305daab8347b42&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:54.042019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.077427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508ab6-05c4-4f1d-a460-36afe645334d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.113030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.114200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.114474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.114739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.122729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.124291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:50:54.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:54.137941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 353/694] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:50:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:54.138918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.146624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d6818-28b3-4427-8697-16404f3dd8f9 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.181215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.181847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.182119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.182400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.182981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.184357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:54.193829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:54.194862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:54.195716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 342/695] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:54.196985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.224265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.251200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.251844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae416fd2-a53e-4ac1-ab58-4f0ddb464d72 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.286277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.287078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.287188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.287367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.288048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.289354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.295316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 354/696] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 16:50:54.296126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.322240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2b12d-c838-4773-a963-dfd1091c34e8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.375198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.376503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.376989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.377767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.379798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:54.380392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.382050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.408634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 343/697] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:54.414837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.420417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be88e7-d2ec-4b9f-a58f-f1c01338f071 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.470113 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.471357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.471837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.472550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.473955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:54.474266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.476060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.482929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 355/698] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:54.489584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.492958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.538166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839edf-8fc8-4838-8ed9-1dc08f021ce6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.557348 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.575194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58335237-a8c2-4529-ba69-1bc707f47deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.575804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58335237-a8c2-4529-ba69-1bc707f47deb None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.576121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58335237-a8c2-4529-ba69-1bc707f47deb None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.576398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58335237-a8c2-4529-ba69-1bc707f47deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.603276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e361-71bd-4d37-842a-e3bc75eda5b1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.654009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.654629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.654906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.655193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.655964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:54.656310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.658317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.659381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58335237-a8c2-4529-ba69-1bc707f47deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.661795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58335237-a8c2-4529-ba69-1bc707f47deb None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.669371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 356/699] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:54.673165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.746768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.792159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 344/700] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 16:50:54.792159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199af826-0f49-4138-bef1-ec442bf3e7bc None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.792159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199af826-0f49-4138-bef1-ec442bf3e7bc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.792159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199af826-0f49-4138-bef1-ec442bf3e7bc None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.792159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 345/701] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:54.793073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199af826-0f49-4138-bef1-ec442bf3e7bc None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.799663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9759d4-1215-4471-9596-cfeeaae8a269 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.803670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:54.838939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.839547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.839827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.840129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.840836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] RBAC: Adding query filter params (interface=public, service_id=fa7c11ee6c2742c0956b9c5aef9fa921, region_id=RegionOne) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:54.841147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.842532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f07c-a475-4e3e-86c3-8a7bdc36aff9 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.854828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 357/702] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/endpoints?service_id=fa7c11ee6c2742c0956b9c5aef9fa921&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:54.854828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.854828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.854828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.874794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:54.875984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:54.909568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199af826-0f49-4138-bef1-ec442bf3e7bc None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:54.947228 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 358/703] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:54.949001 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:54.950529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:54.951165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:54.951448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:54.951722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:54.963889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:54.965356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:50:54.980342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:54.981347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:50:54.982209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:50:54.982209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:50:54.982209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:54.982209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 346/704] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:54.987197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.025655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:55.056699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:55.066489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214b0eb-07e0-47e6-af9f-840bbacf69ff None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.102204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.102668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d783-7103-40dc-8107-e5272fa10ee8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.102989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.103338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.103682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.104405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.105832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.112728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 359/705] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 16:50:55.146789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.147703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.148224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.148640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.149530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:55.149984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.151592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.165746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 347/706] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:55.170949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.232716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.289472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e79ff-06dc-4d35-a3b1-10fe43862ff2 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.315924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:55.332790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.333432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.333712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.333983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.341406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.342995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.351904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efe6d3-c40b-4b42-858d-336006c38f6f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:50:55.356452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:55.357334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:55.358225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 360/707] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:55.364886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.383472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:55.400105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.400880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.401236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.401604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.402280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.403992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:55.412965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:55.414197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:50:55.415207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:55.417796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 348/708] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:50:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:50:55.423526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.492865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a977f0-204d-4329-ae46-466bf5ec94be None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.530481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972efbb8-44a4-4e56-82fd-9c513a72a592 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.538114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.538920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.539783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.540270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.541111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:55.541923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.544229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.565116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 361/709] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:55.566221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.573759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.574478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.574850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.575215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.575942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] RBAC: Adding query filter params (name=compute) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:55.576362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.577963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.588151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 349/710] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:55.589712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:55.630367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:55.645895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:55.701922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82be8-fe35-4f3e-a131-120edd5c11bb None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.705877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7139a-3545-42ea-b3ef-42a6b478afab None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:55.744062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.745015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.745353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.745724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.746509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:55.746927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.748415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23c01f-3054-45e7-a6a1-5f32c526b21c None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.749954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:55.750711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:55.751496 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:55.751919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:55.752794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] RBAC: Adding query filter params (role.id=49f33a0fd78d4b95aef3f4a074616709, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=c5b6addb805b4d0a89ee2cd9ac3de27a) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:55.754674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:55.755341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b505d8-70f5-47a0-b928-47ca43d8bee8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:55.760742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 350/711] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:50:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:55.768190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 362/712] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:50:55 2022] GET /identity/v3/role_assignments?user.id=c5b6addb805b4d0a89ee2cd9ac3de27a&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=49f33a0fd78d4b95aef3f4a074616709 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:56.534322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341806-4d40-4663-aa07-62971021a3ae None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:56.534843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341806-4d40-4663-aa07-62971021a3ae None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:56.535171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341806-4d40-4663-aa07-62971021a3ae None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:56.537265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 351/713] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:56.542793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:56.543211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:56.543385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:56.562341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:56.563510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:56.632007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 363/714] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:56.902286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd341806-4d40-4663-aa07-62971021a3ae None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:57.008871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd341806-4d40-4663-aa07-62971021a3ae None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:57.024522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:57.045855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:57.046492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.046772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.047045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.047630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:57.049072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:57.058559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:57.059763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 16:50:57.060681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:57.066601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 352/715] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:57.067953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:57.107824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:57.169611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2364a6-8e2a-4164-86c7-25b7d26671de None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:57.204549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:57.205159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.205437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.205708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.206335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:57.206636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:57.208604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:57.218017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 364/716] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:57.219385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:57.330988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee223d-1928-4265-b155-1d1685eb5754 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:57.367532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:57.368167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.368444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.368719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.369345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:57.369683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:57.371091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9e1a6-9fe2-4ac4-97ec-f2b773810284 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:57.376582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 353/717] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 10 16:50:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:57.382808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:57.410870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa3666-a04d-4525-a34b-3dd17ac5b77f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.411081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa3666-a04d-4525-a34b-3dd17ac5b77f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.411304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa3666-a04d-4525-a34b-3dd17ac5b77f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.413946 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 354/718] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:57.417946 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.418230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.418506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.437008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:57.438181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:57.477196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:57.489871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc909b2a-ad59-4cae-a994-2a296053fb63 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:57.491779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:57.520326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 355/719] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:57.531013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:57.531797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.532315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.532730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.533606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:57.535209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:57.599579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 365/720] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:50:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:57.798472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:57.908997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569d415-2d04-41e3-b21b-2d994a5e28cf None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:57.952480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:57.953130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:57.953400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:57.953661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:57.954254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:57.955651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-aa40f602-4346-41e6-ae77-19879281d3d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:57.965510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:57.966763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:50:57.967793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 356/721] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:50:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:57.975469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.001536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eb59a-845b-44f9-b9ba-e7fef1552f11 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.001866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eb59a-845b-44f9-b9ba-e7fef1552f11 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.002096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eb59a-845b-44f9-b9ba-e7fef1552f11 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.003689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 357/722] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:50:58.008848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0dd5-c7ec-4577-8613-4cb8e28ca92d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.009135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0dd5-c7ec-4577-8613-4cb8e28ca92d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.009416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0dd5-c7ec-4577-8613-4cb8e28ca92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.032375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4dfa0dd5-c7ec-4577-8613-4cb8e28ca92d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:58.034426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4dfa0dd5-c7ec-4577-8613-4cb8e28ca92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:58.068058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:58.089528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18719f3c-d6e4-4387-99c0-e170a0b98f94 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.124416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 358/723] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:50:58.125986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b12d3-c20a-4ba8-bae3-03c5dfd1ce20 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.126411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b12d3-c20a-4ba8-bae3-03c5dfd1ce20 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.126821 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b12d3-c20a-4ba8-bae3-03c5dfd1ce20 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.128469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 359/724] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:58.132400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.133568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.134331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.134758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.135176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.135621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.136059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.136826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:58.139817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.141446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.147002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 366/725] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:58.152487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.159476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:58.161424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:58.216456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:58.245153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 360/726] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:50:58.280295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933525da-81f4-4d00-93b0-522971dd336b None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.315855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.316887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.317387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.317852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.318733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:58.319229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.320802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.326394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 367/727] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:50:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:50:58.331678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.397029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.424967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:58.434739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f16686-f04a-48b8-8612-ec21fcaeef54 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.470011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.470621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.470901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.471172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.471832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.473884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.494032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:58.508930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363bbc1-49de-41b0-b7e5-8441a2e6073e None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.547472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 361/728] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:50:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:50:58.554126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.555059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.555553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.558760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] PATH_INFO: `/v3/services/image` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.558760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.558760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.559837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:50:58.569312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:50:58.570419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:50:58.571272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 368/729] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Oct 10 16:50:58 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:50:58.577063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.699846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80e23-7698-43bb-8215-1a4c804e2a5a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.728284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa72f-8199-4d18-96b8-b3a1ec8d69ab None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.746960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.747578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.747861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.748204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.748840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] RBAC: Adding query filter params (name=image) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:58.749144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.750504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.760197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 369/730] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:58.762016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:50:58.774222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc311cdf-fc51-4834-84d6-b699dfb66656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.775243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:58.775471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311cdf-fc51-4834-84d6-b699dfb66656 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.775758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311cdf-fc51-4834-84d6-b699dfb66656 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.776229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311cdf-fc51-4834-84d6-b699dfb66656 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.778073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc311cdf-fc51-4834-84d6-b699dfb66656 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.778498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc311cdf-fc51-4834-84d6-b699dfb66656 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.871337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f0ea9-4582-4329-82b9-4f2b3f22fa2c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:50:58.882393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 362/731] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:50:58 2022] POST /identity/v3/services => generated 242 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:50:58.928124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:50:58.928816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:58.929098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:58.929367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:58.930004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] RBAC: Adding query filter params (type=image) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:50:58.930308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:50:58.932089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6a81e-a944-48aa-b955-91912f4c1159 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:50:58.938057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 370/732] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:50:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:50:59.841033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea88d0-575a-4f0e-b7ba-b01772d693a1 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:59.841773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea88d0-575a-4f0e-b7ba-b01772d693a1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:59.842281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea88d0-575a-4f0e-b7ba-b01772d693a1 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:59.844162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 363/733] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:50:59.849998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:50:59.850812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:50:59.851279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:50:59.879481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:50:59.880994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:50:59.945497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:50:59.950463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 371/734] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:50:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:00.223961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea88d0-575a-4f0e-b7ba-b01772d693a1 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:00.233792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:00.335323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea88d0-575a-4f0e-b7ba-b01772d693a1 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:00.370549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:00.371370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:00.371801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:00.372244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:00.373059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:00.374609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:00.377342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING keystone.server.flask.common [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 10 16:51:00.421038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.sql.core [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 16:51:00.421038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 16:51:00.421038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [parameters: {'id': '4a012f6b6f9141ea990cc9686980ddc7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 10 16:51:00.421038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81514) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4a012f6b6f9141ea990cc9686980ddc7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4a012f6b6f9141ea990cc9686980ddc7. Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:51:00.543694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result.read() Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 10 16:51:00.544726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application yield resource Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise self.value Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application session.commit() Oct 10 16:51:00.545947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.session.flush() Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application compat.raise_( Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 16:51:00.547022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 16:51:00.547905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result.read() Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:51:00.548720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [parameters: {'id': '4a012f6b6f9141ea990cc9686980ddc7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:00.549610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:00.550399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4a012f6b6f9141ea990cc9686980ddc7. Oct 10 16:51:00.551835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:00.554089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 364/735] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Oct 10 16:51:00 2022] POST /identity/v3/projects => generated 262 bytes in 330 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:00.560155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:00.660995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba8e2e-2593-48d0-87c2-13fa178844b6 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:00.702680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:00.703348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:00.703630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:00.703906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:00.712160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:00.713636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-085f46f6-3268-4c5d-932c-de1d0968c0be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:51:00.726576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:00.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:00.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 372/736] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:51:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:00.737654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:00.803808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:00.827580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2008ab-b1a1-490b-b7de-d72d45f59606 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:00.827864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2008ab-b1a1-490b-b7de-d72d45f59606 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:00.828188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2008ab-b1a1-490b-b7de-d72d45f59606 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:00.829767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 373/737] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:00.835173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5271d-34ec-4f94-8c3c-a4b3bf2bac08 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:00.835173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5271d-34ec-4f94-8c3c-a4b3bf2bac08 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:00.835425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5271d-34ec-4f94-8c3c-a4b3bf2bac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:00.850730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fa5f-669a-4ead-a3e3-281871b9a026 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:00.854909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c4e5271d-34ec-4f94-8c3c-a4b3bf2bac08 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:00.856139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c4e5271d-34ec-4f94-8c3c-a4b3bf2bac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:00.887789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:00.888429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:00.888781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:00.889055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:00.889717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:00.890014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:00.891634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60eacd-c21c-49f2-9cc3-a09def0c233a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:00.912269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 365/738] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:51:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:00.932565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:00.946640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 374/739] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:51:01.123689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c57d-2d19-435f-a1ad-49f8dc2d9a6e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.123986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c57d-2d19-435f-a1ad-49f8dc2d9a6e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.124198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c57d-2d19-435f-a1ad-49f8dc2d9a6e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.125542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 366/740] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:01.130960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.131376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.131755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.152431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:01.153687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:01.168622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb6d6-f34f-45e2-ab67-326fd1b39298 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.169355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb6d6-f34f-45e2-ab67-326fd1b39298 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.169635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb6d6-f34f-45e2-ab67-326fd1b39298 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.171769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 367/741] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:01.177951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.178239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.178512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.202213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:01.203764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:01.239030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 375/742] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:01.239885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.287658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 368/743] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:01.350110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47c521-d00a-4b0b-a9ae-4d0abd96ea73 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:01.371310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:01.396418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:01.396418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.396418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.396418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.396812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:01.396812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:01.399751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:01.400738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:51:01.401589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:01.403897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 376/744] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:01.412591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.463103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:01.510060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ce1f5-c4c2-4156-bd3e-bff9cbd9e1d0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:01.529660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.544545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:01.545161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.545440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.545854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.546433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:01.546741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:01.548226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:01.554432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 369/745] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:01.560488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.642421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666507a-f38d-485a-8fb7-725564ac6b3a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:01.672792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f6d88-17ea-4db8-b86f-97e6278bf4b2 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:01.678660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:01.679681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.680036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.680342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.681195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:01.682686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:01.689367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:01.693132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:01.694118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:01.694960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 377/746] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:01.699783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.711916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:01.712604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.712841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.713109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.713736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:01.714033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:01.715374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:01.721256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 370/747] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:01.722032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.756240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:01.810743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a5fcd-bdea-4302-9d3c-b3f2b727410d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:01.826402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135840c-ce34-44f8-9af7-6259995f9504 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:01.845281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:01.845889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.846192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.846440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.847090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:01.848563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:01.860816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:01.861425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:01.861706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:01.861978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:01.862603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:01.863477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:01.864462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:01.869702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 371/748] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:01.872783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.919885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:01.953598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 378/749] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:51:01 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:01.955044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:01.989159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132b5c-fd5b-47ca-964c-47d4531a0e47 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:02.049741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:02.051535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:02.052053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:02.052781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:02.054718 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] RBAC: Adding query filter params (interface=admin, service_id=fa7c11ee6c2742c0956b9c5aef9fa921, region_id=RegionOne) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:02.055257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:02.057135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:02.065244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 372/750] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/endpoints?service_id=fa7c11ee6c2742c0956b9c5aef9fa921&interface=admin®ion_id=RegionOne => generated 193 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:02.071239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:02.098556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323710d-9b21-449c-bf9f-48318786afed None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:02.112280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:02.148978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:02.149607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:02.149920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:02.150196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:02.150835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:02.151141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:02.152658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:02.158002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 379/751] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:51:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:02.164696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:02.189917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4676549-f064-4095-bef3-1fd3833bc945 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:02.195400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:02.235269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:02.235962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:02.236400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:02.236676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:02.237544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:02.240283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:02.250010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 373/752] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:51:02 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 16:51:02.290427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8cc16-98bf-4554-9089-1e6899bb6f87 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:02.330398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:02.331025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:02.331306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:02.331579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:02.332356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] RBAC: Adding query filter params (interface=public, service_id=32ce802abbc3468d9bcb766628a34741, region_id=RegionOne) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:02.332668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:02.334050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332cefb-308d-491d-8c35-a16a2e227002 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:02.340408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 380/753] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:51:02 2022] GET /identity/v3/endpoints?service_id=32ce802abbc3468d9bcb766628a34741&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:02.345996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:02.457715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce2ce-6184-4b6b-a48c-7bc2d468f401 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:02.494116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62565a09-e50f-417d-8134-47a3a93f9084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:02.494703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62565a09-e50f-417d-8134-47a3a93f9084 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:02.495040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62565a09-e50f-417d-8134-47a3a93f9084 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:02.495359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62565a09-e50f-417d-8134-47a3a93f9084 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:02.495967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62565a09-e50f-417d-8134-47a3a93f9084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:02.497530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62565a09-e50f-417d-8134-47a3a93f9084 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:02.504802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 374/754] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:51:02 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 16:51:03.194244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e580e5-94f9-4ecd-8d49-f220fdcdf7de None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:03.194795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e580e5-94f9-4ecd-8d49-f220fdcdf7de None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:03.194795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e580e5-94f9-4ecd-8d49-f220fdcdf7de None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:03.196443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 381/755] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:03.201747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:03.202037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:03.202314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:03.221246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:03.222429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:03.250403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:03.294414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 375/756] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:03 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:03.561910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e580e5-94f9-4ecd-8d49-f220fdcdf7de None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:03.601555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:03.668564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e580e5-94f9-4ecd-8d49-f220fdcdf7de None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:03.708662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:03.709274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:03.709553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:03.709816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:03.710469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:03.711879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:03.714496 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING keystone.server.flask.common [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 10 16:51:03.757784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.sql.core [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 16:51:03.757784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 10 16:51:03.757784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [parameters: {'user_id': 'fa0edebdda164554bacde7dc6452eac3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 10 16:51:03.757784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81513) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result.read() Oct 10 16:51:03.765856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:51:03.766910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application yield resource Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise self.value Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application session.commit() Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 16:51:03.767838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.session.flush() Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application compat.raise_( Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 16:51:03.768729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:51:03.769620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application result.read() Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:51:03.770444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa0edebdda164554bacde7dc6452eac3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:03.771246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 10 16:51:03.772014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:03.775105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 382/757] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 10 16:51:03 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:03.775999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:03.873210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89adce-099d-4878-a034-b1c3585b3fff None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:03.907868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:03.908527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:03.908806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:03.909078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:03.916416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:03.917843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-112ef61d-d586-431b-93a1-c6b35780cb97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:51:03.929938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:03.931142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 16:51:03.932102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:03.936095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 376/758] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:03.937700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:04.037107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:04.041213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea1445-d4f4-4c75-bffc-c5fcc0127f30 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:04.079600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:04.080271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.080549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.080820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.081585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:04.081903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:04.083695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5bbaa-df97-4360-accc-f89242adc872 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:04.125117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 383/759] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 10 16:51:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:04.343416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046388-0044-4ccb-be2e-fbec4796bfe0 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.343902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046388-0044-4ccb-be2e-fbec4796bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.344447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046388-0044-4ccb-be2e-fbec4796bfe0 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.346326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 377/760] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:04.352522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b90ca-419b-4310-b499-f0be521416cf None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.353396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b90ca-419b-4310-b499-f0be521416cf None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.353887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b90ca-419b-4310-b499-f0be521416cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.373264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-cb6b90ca-419b-4310-b499-f0be521416cf None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:04.375088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cb6b90ca-419b-4310-b499-f0be521416cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:04.452361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 384/761] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:04 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:04.525758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371e0b7-3cf8-4a8b-8180-99db51882783 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.526041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371e0b7-3cf8-4a8b-8180-99db51882783 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.526491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371e0b7-3cf8-4a8b-8180-99db51882783 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.527833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 378/762] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:04.533266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d660d-649d-4a05-8847-4e964bb1d361 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.533558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d660d-649d-4a05-8847-4e964bb1d361 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.533834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d660d-649d-4a05-8847-4e964bb1d361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.552423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-570d660d-649d-4a05-8847-4e964bb1d361 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:04.553599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-570d660d-649d-4a05-8847-4e964bb1d361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:04.626581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 385/763] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:04 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:04.685218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176bd47-1b82-4926-8e14-b0c4fac94103 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.685502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176bd47-1b82-4926-8e14-b0c4fac94103 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.685955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176bd47-1b82-4926-8e14-b0c4fac94103 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.687429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 379/764] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:04.692916 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.693208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.693485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.718563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:04.718563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:04.719457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176bd47-1b82-4926-8e14-b0c4fac94103 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:04.727903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:04.744596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:04.788619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 386/765] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:04 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:04.829240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176bd47-1b82-4926-8e14-b0c4fac94103 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:04.869650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:04.870550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:04.871008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:04.871456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] PATH_INFO: `/v3/services/image` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:04.872318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:04.873670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:04.886423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:04.887413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:04.889263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 380/766] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Oct 10 16:51:04 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:04.893803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:04.895144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:04.985738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.000676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2526b-cef3-49ef-9a66-a72953502f87 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.002840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50307c0b-eaaf-42a2-a7f5-43c5af9d8f21 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.039474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.040127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.040404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.040592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.040707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.041195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.041297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.041477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.041744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.042366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] RBAC: Adding query filter params (name=image) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.042580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.042686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.044000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:05.052120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:05.053117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 16:51:05.053950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:05.055303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 387/767] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:05.056644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.058322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 381/768] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 10 16:51:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:05.059005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.070579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.155693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3adf-f43b-44c2-81c9-58a3d81a021d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.156365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416412-c317-455b-be05-d1f39f1052d5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.188122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.188718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.188853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.188994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.189262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.189352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.189622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.189885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.189979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.190182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.190511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.190815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.192088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.192728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.196753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 388/769] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:05.202122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 382/770] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:05.202254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.202772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.220510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.273541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.297391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a20f8-22c6-496e-b8b2-020ff79aede5 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.302566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1345163-0a34-44d9-a7af-e9039f173c36 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.332653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.333276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.333555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.333823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.334400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.335749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.335961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.336593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.336874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.337145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.337836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] RBAC: Adding query filter params (interface=public, service_id=b136411e57ca42afbecb4365e867573e, region_id=RegionOne) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.338144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.339483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:05.344559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:05.348425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 16:51:05.349526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:05.350461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 383/771] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/endpoints?service_id=b136411e57ca42afbecb4365e867573e&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:05.350461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.350590 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 389/772] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:05.350590 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.424204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.446659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7758a-e3ce-45b6-aae9-1e5c28040ca1 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.447245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d5973-97ac-42a7-8073-53971c1eff34 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.479723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.480347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.480578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.480712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.480879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.481230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.481495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.481601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.481770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.481858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.482363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.483120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.483689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.487856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 384/773] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:05.489050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.490572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 390/774] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services => generated 1957 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 16:51:05.494153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.496957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.607947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5767c-ecb3-4efd-b3fb-cb9099b14ca4 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.610750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fb268-b581-42cb-8b22-2d84577afb35 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.648326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.649056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.649340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.649638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.650218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.650335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.650548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.650832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.651109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.651377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.651920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.652109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.653505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.657579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 385/775] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:05.664486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.673830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.751656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 391/776] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:51:05 2022] POST /identity/v3/endpoints => generated 325 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:05.765806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926eb43-386f-47ee-8ee0-0c8e3b5716bc None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.812469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.813094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.813520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.813839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.814487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:05.814867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.816208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c193a-91af-438d-85fc-7c33dcc890a5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:05.822514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 386/777] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 10 16:51:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:05.828342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:05.906853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:05.950065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033a1fb-5fc0-42ea-bb1c-41296a154024 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:05.985208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c76b6-68ce-4e40-96f0-e881cc8562ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:05.985874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c76b6-68ce-4e40-96f0-e881cc8562ac None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:05.986161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c76b6-68ce-4e40-96f0-e881cc8562ac None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:05.986432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c76b6-68ce-4e40-96f0-e881cc8562ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:05.987146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c76b6-68ce-4e40-96f0-e881cc8562ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:05.989052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c76b6-68ce-4e40-96f0-e881cc8562ac None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:06.084835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 392/778] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:51:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:07.648507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410882f4-b63a-4eff-9eb0-cb334cc5bba5 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:07.649067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410882f4-b63a-4eff-9eb0-cb334cc5bba5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:07.649067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410882f4-b63a-4eff-9eb0-cb334cc5bba5 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:07.650549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 387/779] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:07.655520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a83bc-636c-4d7a-91c1-556c6e7aeab2 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:07.655900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a83bc-636c-4d7a-91c1-556c6e7aeab2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:07.656083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a83bc-636c-4d7a-91c1-556c6e7aeab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:07.674764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-858a83bc-636c-4d7a-91c1-556c6e7aeab2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:07.675975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-858a83bc-636c-4d7a-91c1-556c6e7aeab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:07.749078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 393/780] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:07.924059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c50c-f479-4ab1-a4f1-b526612b4b39 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:07.924399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c50c-f479-4ab1-a4f1-b526612b4b39 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:07.924618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c50c-f479-4ab1-a4f1-b526612b4b39 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:07.926133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 388/781] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:07.931407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:07.931699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:07.931977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:07.950631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:07.951799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:07.991125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:08.017255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2c50c-f479-4ab1-a4f1-b526612b4b39 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.018283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 394/782] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:08.086815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:08.121923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2c50c-f479-4ab1-a4f1-b526612b4b39 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.156630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.157258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.157538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.157811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] PATH_INFO: `/v3/services/image` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.158386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:08.159741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:08.168603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:08.169581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:08.173063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 389/783] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Oct 10 16:51:08 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:08.184100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.284252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c326f-71ff-48f8-bba4-99bf62c2dc01 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.284998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.320128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.320750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.321024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.321316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.321941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] RBAC: Adding query filter params (name=image) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:08.322244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:08.323600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1e387-93b4-464a-a8c9-f19d50713afe None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:08.328522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 395/784] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:08.329660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04a8e2-1e74-40b7-85d4-73626ed408e1 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.329940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04a8e2-1e74-40b7-85d4-73626ed408e1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.330211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04a8e2-1e74-40b7-85d4-73626ed408e1 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.331584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 396/785] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:08.336317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04a8e2-1e74-40b7-85d4-73626ed408e1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.376076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:08.387635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb71b8-eb52-4b5d-97b2-7b9141773b55 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.415571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:08.421264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783baa5-623a-4cf7-9d29-36f2f211e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.421883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783baa5-623a-4cf7-9d29-36f2f211e94f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.422155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783baa5-623a-4cf7-9d29-36f2f211e94f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.422423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783baa5-623a-4cf7-9d29-36f2f211e94f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.423528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783baa5-623a-4cf7-9d29-36f2f211e94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:08.426179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783baa5-623a-4cf7-9d29-36f2f211e94f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:08.427061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 390/786] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:08.429306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.429592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.429881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.437444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04a8e2-1e74-40b7-85d4-73626ed408e1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.449280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:08.450636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:08.471677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.472309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.473100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.473368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.473990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:08.474289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:08.475627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:08.480315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 397/787] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:08.481051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.520978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 391/788] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:08.522192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.585167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f963b7-e573-4781-81f5-ef272d1985f8 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.587530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:08.619930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f295c-7485-47cf-9ec4-7b88273b6cb1 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.621843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.622563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.622839 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.623110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.624063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:08.625708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:08.646297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:08.653892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.659202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.659202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.659202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.659202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:08.659202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:08.711807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 398/789] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:51:08 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:08.743065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 392/790] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:51:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:08.788379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:08.915246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5b344-0f01-4685-be23-95987f15fddd None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:08.957901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:08.958536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:08.958815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:08.959082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:08.971495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:08.976849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:08.977943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:08.978781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 399/791] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:51:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:08.985929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:09.086437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2242-1b9d-476e-a3da-720534644c80 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:09.119957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:09.120588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:09.120864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:09.121133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:09.121787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:09.122087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:09.123428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:09.131818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 393/792] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 10 16:51:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:09.137774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:09.151017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:09.238250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6a08-1738-4a2b-ac2b-4d4e3ac34053 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:09.272274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:09.272904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:09.273180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:09.273446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:09.285145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:09.286586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:51:09.300642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:09.301585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:51:09.302472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:51:09.302472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:09.302472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:09.302472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 400/793] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:09.309253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:09.356010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:09.408433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32089b3-3cac-4e96-9454-8a1d7fa69105 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:09.441757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:09.442360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:09.442634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:09.442896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:09.443523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:09.443817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:09.445185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:09.462011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 394/794] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:09.463587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:09.524971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:09.564736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b108c-1bd6-42da-b2fa-ab71d3696101 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:09.599642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:09.600673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:09.601183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:09.601655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:09.609419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:09.611057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:51:09.622873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:09.623873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:51:09.624844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:09.626856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 401/795] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 10 16:51:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:09.632789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:09.737759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f672e1c-c70a-4c80-add0-820b18932245 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:09.746856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:09.772598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:09.773202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:09.773512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:09.773824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:09.774882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:09.775689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:09.777656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:09.794493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 395/796] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 10 16:51:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:09.800055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:09.907806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afa831-1753-4bcc-9463-9e60c8b5b65d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:09.930957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:09.942727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:09.943354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:09.943641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:09.943915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:09.954083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:09.955581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:51:09.969802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:09.970976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:51:09.972124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:51:09.972124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:09.972124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:09.979649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 402/797] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:09.980472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:10.079735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9fe03-680d-447b-8f42-ff22be696d89 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:10.112662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:10.113822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.114093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.114356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.114979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:10.115276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:10.116635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:10.127618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 396/798] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:10.133010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:10.231243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b24518-5325-4e16-8b8c-5c742e48d7bf None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:10.263670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:10.264306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.264583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.264850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.271174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:10.272620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:51:10.283701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:10.284633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:51:10.286056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:10.288093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 403/799] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:10.291107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:10.316618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:10.390829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df4edb-d34c-4e3c-8f6e-d7a3baa6f71f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:10.424284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:10.424885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.425156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.425423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.426130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:10.426426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:10.427825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97167de-a9c4-4ab8-9405-d041163b405b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:10.439466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 397/800] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:10.445221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:10.481428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:10.548354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ac77c-1a32-4acd-b12a-355182b365dd None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:10.582537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49eb8a-bf18-49d8-8b8a-5a81c7463be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:10.583196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49eb8a-bf18-49d8-8b8a-5a81c7463be6 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.583475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49eb8a-bf18-49d8-8b8a-5a81c7463be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.583748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49eb8a-bf18-49d8-8b8a-5a81c7463be6 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/54179215880d4ed291d18944a2f86fdc/roles/49f33a0fd78d4b95aef3f4a074616709` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.585894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49eb8a-bf18-49d8-8b8a-5a81c7463be6 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=54179215880d4ed291d18944a2f86fdc, role_id=49f33a0fd78d4b95aef3f4a074616709)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:10.587304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49eb8a-bf18-49d8-8b8a-5a81c7463be6 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:10.660741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 404/801] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:51:10 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/54179215880d4ed291d18944a2f86fdc/roles/49f33a0fd78d4b95aef3f4a074616709 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:51:10.944214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1acf8-0972-42f6-bc6a-7192e39da930 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.944731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1acf8-0972-42f6-bc6a-7192e39da930 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.945197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1acf8-0972-42f6-bc6a-7192e39da930 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.947801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 398/802] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:10.953093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.953525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.953693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.972643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:10.973843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:10.987459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7677f-9a8c-49e0-8fb8-9e884b234b0f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.987742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7677f-9a8c-49e0-8fb8-9e884b234b0f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.988055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7677f-9a8c-49e0-8fb8-9e884b234b0f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:10.989521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 399/803] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:51:10.994404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:10.994697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:10.994976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:11.013991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:11.014654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:11.047408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 405/804] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:11.088758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 400/805] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:51:11.319904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:11.344293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:11.371064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:11.387456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:11.426955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30297ab-f7fc-4734-8e1f-a2905f47b410 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:11.462790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:11.463429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:11.463706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:11.463973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:11.465091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:11.466493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:11.468224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 406/806] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:11.473980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e7a12-9df4-45c0-9d72-3c4eba8de027 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:11.474107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:11.512003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:11.512626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:11.512899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:11.513166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:11.513834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:11.515179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:11.556577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.sql.core [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 10 16:51:11.556577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 16:51:11.556577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [parameters: {'id': '0264cbae735c44deafd919f611923e3b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 10 16:51:11.556577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81514) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:51:11.564162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result.read() Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 10 16:51:11.566983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application yield resource Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application next(self.gen) Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise self.value Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application session.commit() Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 16:51:11.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.session.flush() Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application compat.raise_( Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 16:51:11.568768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 16:51:11.569642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application conn.query(q) Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application result.read() Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 16:51:11.570429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [parameters: {'id': '0264cbae735c44deafd919f611923e3b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 10 16:51:11.571291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:11.573266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 401/807] 10.222.0.42 () {60 vars in 1137 bytes} [Mon Oct 10 16:51:11 2022] POST /identity/v3/domains => generated 190 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:11.574343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:11.577232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:11.589118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bea91c-79b3-44c4-9a99-156f2ab277de None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:11.625772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:11.626472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:11.626755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:11.627029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:11.627673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:11.629212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:11.652446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:11.673308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71994b-7f1a-4d0c-a436-e7dfa5200428 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:11.705696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:11.706391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:11.706670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:11.706937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:11.717093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:11.718503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:11.729663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 407/808] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:51:11 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-f203d5a8-0f51-4780-932f-fc581c96378b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:51:11.735313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:11.736367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:51:11.743521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:51:11.743521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:11.743521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:11.743521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 402/809] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:11.743762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:11.857171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59378ff8-6a8b-43ca-b909-bacb3560165c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:11.900697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:11.901320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:11.901597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:11.901869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:11.902519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:11.902817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:11.904213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0ea05-b420-4b50-b1a8-1b324c551b2a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:11.919460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 408/810] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:12.803820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0792c-67f3-44ef-8e34-067695965aa7 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:12.804383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0792c-67f3-44ef-8e34-067695965aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:12.804383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0792c-67f3-44ef-8e34-067695965aa7 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:12.805993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 403/811] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:12.811257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:12.811662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:12.811808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:12.829367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:12.830534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:12.833009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:12.900397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 409/812] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:51:13.165816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0792c-67f3-44ef-8e34-067695965aa7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:13.268359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0792c-67f3-44ef-8e34-067695965aa7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:13.295643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:13.301396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:13.302015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.302294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.302565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.312579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:13.317964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:13.319157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:13.320109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 404/813] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 10 16:51:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:13.325066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:13.419701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f66a4b-12ea-4d3e-816c-0012b2804cd6 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:13.453056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:13.453656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.453934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.454203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.454858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:13.455157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:13.456541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:13.461524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:13.464782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 410/814] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 10 16:51:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:13.470605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:13.568385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48866-034b-466a-aacb-bef9d8f3683a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:13.601053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:13.601674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.601958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.602228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.611973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:13.613976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:51:13.627998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:13.629678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:51:13.630554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:51:13.630554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:13.630554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:13.630554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 405/815] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:13.635395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:13.731841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf420f5-5dd6-4408-9ee7-b90652fd99b3 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:13.764708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:13.765314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.765596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.765870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.766508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:13.766813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:13.768210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbd5f7-4e97-4e52-84fa-f495aac8214c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:13.779714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 411/816] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:13.785882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:13.831995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:13.890416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f722c-d056-40a2-b4dc-c3e4a85b436a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:13.918321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98870007-f7f8-4087-aa7f-0115304358b8 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.918820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98870007-f7f8-4087-aa7f-0115304358b8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.918923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98870007-f7f8-4087-aa7f-0115304358b8 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.920245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 412/817] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:51:13.923623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:13.924353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.924638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.924914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.925097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:13.925378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:13.926209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:13.933801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:13.935227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:13.944509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:13.945674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:51:13.946137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:13.947218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:13.948187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 406/818] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 10 16:51:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:13.952893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.002268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.013467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 413/819] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 16:51:14.047843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db63be-18ed-494f-9df3-f0d63402cec9 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.049353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.081493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.082613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.082890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.083159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.083873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:14.084217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.085615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.102341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 407/820] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 10 16:51:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:14.108049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.202885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271f2d2-4667-4f36-b49e-c127865fd6f3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.237035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.237637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.237913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.238186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.281899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.293693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.303610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.305209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.379171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb0576-765f-4a3a-a92c-d0adff766a70 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.401795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 414/821] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:51:14.407414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.412957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.413587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.413860 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.414125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.424131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:14.428858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:14.429818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:14.430650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 408/822] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:14.436011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.459395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.495255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.505152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85906328-f54e-4216-8a86-ddbe1574d3c1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.535303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867856-9d1a-4719-8eb2-f5a6b990d490 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.538811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.539443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.539725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.539999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.549738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.551169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:51:14.566572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:14.567517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:51:14.568537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:51:14.568537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:51:14.568537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:14.568537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 415/823] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:14.569605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.570221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.570504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.570794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.571435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:14.571737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.573244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.573383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.586526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 409/824] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:14.587495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.664981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.671314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4ab50-f94d-4e83-85eb-2fc3b1a59263 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.682290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79160172-574c-4618-be2d-f4eb6ba19700 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.704529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.705137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.705419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.705695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.706331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:14.706683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.708589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.716671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.717753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.718029 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.718301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] PATH_INFO: `/v3/users/9f8ad0c9035445fab2ec5b476c82464a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.720057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 416/825] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:14.725613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.732205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.732320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=9f8ad0c9035445fab2ec5b476c82464a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.733737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.735359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 410/826] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/users/9f8ad0c9035445fab2ec5b476c82464a => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:14.741085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.826604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35706c1-051b-46a8-97f1-dfdfaaf1f8d5 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.837355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805422-02ca-49f3-841d-c72f98ce5ab6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.859948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.860621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.860892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.861161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.870296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:14.870718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.870919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:14.871192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:14.871459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] PATH_INFO: `/v3/projects/f66f9f2833084067b56229f9e9b83e01` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:14.872362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.872650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] RBAC: Authorizing `identity:get_project(project_id=f66f9f2833084067b56229f9e9b83e01)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:14.874045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:14.875655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 411/827] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/projects/f66f9f2833084067b56229f9e9b83e01 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:14.881909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:51:14.885376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:14.886312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:14.887205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 417/828] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:14.892047 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:14.893672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.953261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:14.982044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777f21c-bd09-4a05-a4a8-c5c516c7fa58 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:14.996533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b4d58-94c7-4e2d-92e8-a340b1bc89ff None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:15.016971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721451aa-aca7-4806-b69b-7d608349b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:15.017630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721451aa-aca7-4806-b69b-7d608349b3dd None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:15.017902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721451aa-aca7-4806-b69b-7d608349b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:15.018176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721451aa-aca7-4806-b69b-7d608349b3dd None admin] PATH_INFO: `/v3/projects/f66f9f2833084067b56229f9e9b83e01/users/9f8ad0c9035445fab2ec5b476c82464a/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:15.020110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721451aa-aca7-4806-b69b-7d608349b3dd None admin] RBAC: Authorizing `identity:create_grant(project_id=f66f9f2833084067b56229f9e9b83e01, user_id=9f8ad0c9035445fab2ec5b476c82464a, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:15.021631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721451aa-aca7-4806-b69b-7d608349b3dd None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:15.031244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:15.031844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:15.032145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:15.032416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:15.033094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:15.033393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:15.034773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:15.045698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 418/829] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:15.052171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:15.123777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 412/830] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:51:14 2022] PUT /identity/v3/projects/f66f9f2833084067b56229f9e9b83e01/users/9f8ad0c9035445fab2ec5b476c82464a/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:51:15.157993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e57ec-beb3-4315-a0f0-ba8ee7b6b78a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:15.182773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:15.195122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:15.196152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:15.196152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:15.196366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:15.197096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] RBAC: Adding query filter params (role.id=49f33a0fd78d4b95aef3f4a074616709, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=54179215880d4ed291d18944a2f86fdc) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:15.197406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:15.198844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b14ea4-5789-44ff-8cd8-ededfceb8f20 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:15.205407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 419/831] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:51:15 2022] GET /identity/v3/role_assignments?user.id=54179215880d4ed291d18944a2f86fdc&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=49f33a0fd78d4b95aef3f4a074616709 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:17.368204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e0df8-b0b1-4fbf-a0e1-88aa5309c689 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:17.368766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e0df8-b0b1-4fbf-a0e1-88aa5309c689 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:17.368766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e0df8-b0b1-4fbf-a0e1-88aa5309c689 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:17.370100 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 413/832] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:17.375473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:17.375875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:17.376017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:17.395005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:17.396215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:17.464649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 420/833] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:17.727536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e0df8-b0b1-4fbf-a0e1-88aa5309c689 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:17.793240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:17.827290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e0df8-b0b1-4fbf-a0e1-88aa5309c689 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:17.859198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:17.860185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:17.860752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:17.861728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:17.872833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:17.877651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:17.879395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:17.880302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 414/834] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:17.885378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:17.980175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6b519-78f0-4068-8730-e683f01575d5 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:17.996153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:18.013200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:18.013796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:18.014073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:18.014341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:18.014995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:18.015298 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:18.016670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:18.024336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 421/835] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:18.030204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:18.125225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaf6e3-13a1-466b-9e61-59b39ad3469a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:18.150526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:18.158069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:18.161466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:18.161466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:18.161466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] PATH_INFO: `/v3/users/9f8ad0c9035445fab2ec5b476c82464a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:18.161466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] RBAC: Authorizing `identity:get_user(user_id=9f8ad0c9035445fab2ec5b476c82464a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:18.161768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:18.163268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 415/836] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:18 2022] GET /identity/v3/users/9f8ad0c9035445fab2ec5b476c82464a => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:18.168899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:18.264665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02642f-787a-49b3-897a-8aa40a086607 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:18.297639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:18.298226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:18.298500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:18.298800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:18.342428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:18.363823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:18.365300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:18.573899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 422/837] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:51:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 16:51:18.579735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:18.674768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2433-8db7-4753-b7d4-370737a711b6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:18.707348 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96197798-63ef-471b-8ef7-d794f45ff9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:18.707963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96197798-63ef-471b-8ef7-d794f45ff9ac None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:18.708268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96197798-63ef-471b-8ef7-d794f45ff9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:18.708538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96197798-63ef-471b-8ef7-d794f45ff9ac None admin] PATH_INFO: `/v3/projects/f66f9f2833084067b56229f9e9b83e01` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:18.709639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96197798-63ef-471b-8ef7-d794f45ff9ac None admin] RBAC: Authorizing `identity:get_project(project_id=f66f9f2833084067b56229f9e9b83e01)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:18.711051 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96197798-63ef-471b-8ef7-d794f45ff9ac None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:18.712680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 416/838] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:18 2022] GET /identity/v3/projects/f66f9f2833084067b56229f9e9b83e01 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:18.718578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:18.792493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:18.813175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b6d5-6459-4564-8d70-5f27a0396fb3 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:18.845190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:18.845720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:18.845993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:18.846258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:18.846969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=f66f9f2833084067b56229f9e9b83e01, user.id=9f8ad0c9035445fab2ec5b476c82464a) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:18.847272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:18.848693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186598a-0b0c-4b37-bea8-bd84140d56fd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:18.854178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 423/839] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:51:18 2022] GET /identity/v3/role_assignments?user.id=9f8ad0c9035445fab2ec5b476c82464a&scope.project.id=f66f9f2833084067b56229f9e9b83e01&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:20.960886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7af10-072d-43d0-b383-281bb7d6b946 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:20.961390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7af10-072d-43d0-b383-281bb7d6b946 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:20.961502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7af10-072d-43d0-b383-281bb7d6b946 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:20.962775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 417/840] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:20.968003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:20.968420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:20.968580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:20.986667 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:20.987833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:21.061810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 424/841] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:21.334987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7af10-072d-43d0-b383-281bb7d6b946 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:21.396076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:21.432329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7af10-072d-43d0-b383-281bb7d6b946 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:21.465234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7369c-a106-4605-931b-7ae6ac575cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:21.465845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7369c-a106-4605-931b-7ae6ac575cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:21.466117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7369c-a106-4605-931b-7ae6ac575cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:21.466383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7369c-a106-4605-931b-7ae6ac575cc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:21.467474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7369c-a106-4605-931b-7ae6ac575cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:21.468899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7369c-a106-4605-931b-7ae6ac575cc1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:21.470425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 418/842] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:21.476046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:21.542587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:21.571629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acc41-2dff-4bd3-be2a-8927343e9f0b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:21.603879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63b759-d151-4f2d-b876-989d8f923bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:21.604448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b759-d151-4f2d-b876-989d8f923bf6 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:21.604727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b759-d151-4f2d-b876-989d8f923bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:21.604997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b759-d151-4f2d-b876-989d8f923bf6 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:21.605656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63b759-d151-4f2d-b876-989d8f923bf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:21.607054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63b759-d151-4f2d-b876-989d8f923bf6 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:21.698200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 425/843] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:51:21 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:51:23.848432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c5a8e-594a-4e37-a778-cdb692c75864 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:23.848997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c5a8e-594a-4e37-a778-cdb692c75864 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:23.848997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c5a8e-594a-4e37-a778-cdb692c75864 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:23.850293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 419/844] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:23.855489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:23.855862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:23.856066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:23.873886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:23.875068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:23.938801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 426/845] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:24.203326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c5a8e-594a-4e37-a778-cdb692c75864 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:24.299230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c5a8e-594a-4e37-a778-cdb692c75864 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:24.327281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:24.332437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:24.333041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:24.333312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:24.333579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] PATH_INFO: `/v3/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:24.334935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] RBAC: Authorizing `identity:get_role(role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:24.336306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:24.337584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 420/846] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:24 2022] GET /identity/v3/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:24.343162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:24.398541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:24.438332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c594-e784-4e94-ad38-36ada3f61f0a None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:24.471904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:24.472526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:24.472806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:24.473079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] PATH_INFO: `/v3/users/338aa715f14d420395fbbd8b56991338` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:24.486209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] RBAC: Authorizing `identity:get_user(user_id=338aa715f14d420395fbbd8b56991338)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:24.487610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:24.489234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 427/847] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:24 2022] GET /identity/v3/users/338aa715f14d420395fbbd8b56991338 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:24.494945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:24.591875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f5db4-c42e-42da-9532-8e945aa58f78 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:24.624494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2270da-dbdd-42f5-8209-295146f668d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:24.625109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2270da-dbdd-42f5-8209-295146f668d6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:24.625388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2270da-dbdd-42f5-8209-295146f668d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:24.625659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2270da-dbdd-42f5-8209-295146f668d6 None admin] PATH_INFO: `/v3/projects/f66f9f2833084067b56229f9e9b83e01` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:24.626769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2270da-dbdd-42f5-8209-295146f668d6 None admin] RBAC: Authorizing `identity:get_project(project_id=f66f9f2833084067b56229f9e9b83e01)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:24.628767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2270da-dbdd-42f5-8209-295146f668d6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:24.630399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 421/848] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:24 2022] GET /identity/v3/projects/f66f9f2833084067b56229f9e9b83e01 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:24.636211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:24.729934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852556c-adff-45ad-8dbb-322d5d1e7b36 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:24.745793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:24.762965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78ddd6-df8c-4318-bb83-c09052b24d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:24.763606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78ddd6-df8c-4318-bb83-c09052b24d59 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:24.763886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78ddd6-df8c-4318-bb83-c09052b24d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:24.764195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78ddd6-df8c-4318-bb83-c09052b24d59 None admin] PATH_INFO: `/v3/projects/f66f9f2833084067b56229f9e9b83e01/users/338aa715f14d420395fbbd8b56991338/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:24.766294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78ddd6-df8c-4318-bb83-c09052b24d59 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66f9f2833084067b56229f9e9b83e01, user_id=338aa715f14d420395fbbd8b56991338, role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:24.767681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78ddd6-df8c-4318-bb83-c09052b24d59 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:24.847666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 428/849] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:51:24 2022] PUT /identity/v3/projects/f66f9f2833084067b56229f9e9b83e01/users/338aa715f14d420395fbbd8b56991338/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:51:26.963869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcd6c0-ac9c-4bdd-9b44-4281eafe669b None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:26.964439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcd6c0-ac9c-4bdd-9b44-4281eafe669b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:26.964439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcd6c0-ac9c-4bdd-9b44-4281eafe669b None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:26.965802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 422/850] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:26.971139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:26.971536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:26.971693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:26.989869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:26.991052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:27.057967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 429/851] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:27.336375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcd6c0-ac9c-4bdd-9b44-4281eafe669b None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:27.400699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:27.432459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcd6c0-ac9c-4bdd-9b44-4281eafe669b None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:27.465520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:27.466135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:27.466411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:27.466681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] PATH_INFO: `/v3/roles/215077d32ef84d5eaa4d487616aa2ec3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:27.468090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] RBAC: Authorizing `identity:get_role(role_id=215077d32ef84d5eaa4d487616aa2ec3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:27.469438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:27.470743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 423/852] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:27 2022] GET /identity/v3/roles/215077d32ef84d5eaa4d487616aa2ec3 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:27.476423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:27.569962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344aa34-4b59-4489-820c-d562ebb7fcc7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:27.576867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:27.601931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:27.602730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:27.603189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:27.603784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] PATH_INFO: `/v3/users/338aa715f14d420395fbbd8b56991338` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:27.604982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] RBAC: Authorizing `identity:get_user(user_id=338aa715f14d420395fbbd8b56991338)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:27.606388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:27.607958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 430/853] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:27 2022] GET /identity/v3/users/338aa715f14d420395fbbd8b56991338 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:27.614188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:27.709487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab09e0-4cf8-4caf-ab35-29bf6990241a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:27.741320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:27.741906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:27.742177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:27.742442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:27.745715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:27.805653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:27.807367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:27.900543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 424/854] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:51:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 16:51:27.906581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:28.000978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb37244-992b-4e7f-aa46-0f9f6c52ebf4 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:28.033365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653e1d2-5817-425b-9317-be6607677c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:28.033978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653e1d2-5817-425b-9317-be6607677c45 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:28.034257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653e1d2-5817-425b-9317-be6607677c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:28.034554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653e1d2-5817-425b-9317-be6607677c45 None admin] PATH_INFO: `/v3/projects/f66f9f2833084067b56229f9e9b83e01` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:28.035647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653e1d2-5817-425b-9317-be6607677c45 None admin] RBAC: Authorizing `identity:get_project(project_id=f66f9f2833084067b56229f9e9b83e01)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:28.037067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653e1d2-5817-425b-9317-be6607677c45 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:28.038647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 431/855] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:27 2022] GET /identity/v3/projects/f66f9f2833084067b56229f9e9b83e01 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:28.044668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:28.077590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:28.138202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365bc2f-8649-4240-ae36-31b7a20a8398 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:28.170087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:28.170670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:28.170942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:28.171213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:28.171919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] RBAC: Adding query filter params (role.id=215077d32ef84d5eaa4d487616aa2ec3, scope.project.id=f66f9f2833084067b56229f9e9b83e01, user.id=338aa715f14d420395fbbd8b56991338) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:28.172251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:28.173635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7c256-6314-43a3-9b0a-c95cf17b873d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:28.179083 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 425/856] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:51:28 2022] GET /identity/v3/role_assignments?user.id=338aa715f14d420395fbbd8b56991338&scope.project.id=f66f9f2833084067b56229f9e9b83e01&role.id=215077d32ef84d5eaa4d487616aa2ec3 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:30.303525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ead00d-e14d-47b0-96a7-8ef9e448f666 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:30.304170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ead00d-e14d-47b0-96a7-8ef9e448f666 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:30.304264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ead00d-e14d-47b0-96a7-8ef9e448f666 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:30.305587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 432/857] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:30.311102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:30.311536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:30.311631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:30.329541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:30.330724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:30.394722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 426/858] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:30.662478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ead00d-e14d-47b0-96a7-8ef9e448f666 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:30.705369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:30.760170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ead00d-e14d-47b0-96a7-8ef9e448f666 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:30.792820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a6de3-afbd-4466-8705-eb6194b02166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:30.793431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a6de3-afbd-4466-8705-eb6194b02166 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:30.793705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a6de3-afbd-4466-8705-eb6194b02166 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:30.793967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a6de3-afbd-4466-8705-eb6194b02166 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:30.795047 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a6de3-afbd-4466-8705-eb6194b02166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:30.796477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a6de3-afbd-4466-8705-eb6194b02166 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:30.797994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 433/859] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:30.803812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:30.832223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:30.899155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd2e7b-0455-4f71-a91b-1586bf8d9da0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:30.931891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc75763-5dbb-4be1-83b1-fd4d64706ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:30.932521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc75763-5dbb-4be1-83b1-fd4d64706ca6 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:30.932801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc75763-5dbb-4be1-83b1-fd4d64706ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:30.933072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc75763-5dbb-4be1-83b1-fd4d64706ca6 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:30.933725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc75763-5dbb-4be1-83b1-fd4d64706ca6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:30.935134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc75763-5dbb-4be1-83b1-fd4d64706ca6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:31.043414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 427/860] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:51:30 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:51:33.214658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146d5c-08d4-4abd-99ff-e3b930b7c36f None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:33.215181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146d5c-08d4-4abd-99ff-e3b930b7c36f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:33.215181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146d5c-08d4-4abd-99ff-e3b930b7c36f None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:33.216649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 434/861] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:33.222441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:33.222852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:33.222988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:33.241249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:33.242425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:33.306423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 428/862] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:33.572334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146d5c-08d4-4abd-99ff-e3b930b7c36f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:33.668729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146d5c-08d4-4abd-99ff-e3b930b7c36f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:33.679252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:33.701018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac34525-0893-43b9-b17d-a71a7ca0848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:33.701630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac34525-0893-43b9-b17d-a71a7ca0848d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:33.701900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac34525-0893-43b9-b17d-a71a7ca0848d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:33.702165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac34525-0893-43b9-b17d-a71a7ca0848d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:33.703249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac34525-0893-43b9-b17d-a71a7ca0848d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:33.704672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac34525-0893-43b9-b17d-a71a7ca0848d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:33.706681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 435/863] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:51:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:33.712452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:33.726733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:33.806582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca691-0555-49dc-a9d5-1b5f351e0944 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:33.838255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047400-f6b9-4935-8079-d0d0e52133b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:33.838788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047400-f6b9-4935-8079-d0d0e52133b7 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:33.839062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047400-f6b9-4935-8079-d0d0e52133b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:33.839326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047400-f6b9-4935-8079-d0d0e52133b7 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:33.839959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75047400-f6b9-4935-8079-d0d0e52133b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:33.841392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75047400-f6b9-4935-8079-d0d0e52133b7 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:33.933164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 429/864] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:51:33 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:36.058614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d1a4b-d8b8-4535-9fc3-8342e6931e79 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:36.059160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d1a4b-d8b8-4535-9fc3-8342e6931e79 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:36.059160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d1a4b-d8b8-4535-9fc3-8342e6931e79 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:36.060535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 436/865] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:36.066068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:36.066435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:36.066606 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:36.084200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:36.085365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:36.143062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:36.147911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 430/866] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:36.418781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d1a4b-d8b8-4535-9fc3-8342e6931e79 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:36.515750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d1a4b-d8b8-4535-9fc3-8342e6931e79 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:36.547947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:36.548601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:36.548878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:36.549150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:36.559659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:36.564431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:36.565533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:36.566754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 437/867] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:36.571204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:36.666128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d5418-a295-4b94-9f1f-1b8e83d45566 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:36.698516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:36.699059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:36.699341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:36.699618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:36.700304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:36.700610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:36.701963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:36.710237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 431/868] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:36.716423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:36.822118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:36.879376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4b4da-0ac8-46a0-b694-068effd0a613 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:36.912522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:36.913132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:36.913406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:36.913673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] PATH_INFO: `/v3/users/0284ef2a29064c22a1c9e5f3c318224d` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:36.926040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] RBAC: Authorizing `identity:get_user(user_id=0284ef2a29064c22a1c9e5f3c318224d)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:36.927478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:36.929070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 438/869] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:36 2022] GET /identity/v3/users/0284ef2a29064c22a1c9e5f3c318224d => generated 286 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:36.935275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:36.991453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:37.030330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d72e0e-7100-4286-98d9-355f12e163f0 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:37.061745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e7a8a-0738-4543-bbff-fb2f8bad4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:37.062357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e7a8a-0738-4543-bbff-fb2f8bad4794 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:37.062636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e7a8a-0738-4543-bbff-fb2f8bad4794 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:37.062907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e7a8a-0738-4543-bbff-fb2f8bad4794 None admin] PATH_INFO: `/v3/projects/50d95139f7ea44a98749b3cc78ad4e3a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:37.064000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e7a8a-0738-4543-bbff-fb2f8bad4794 None admin] RBAC: Authorizing `identity:get_project(project_id=50d95139f7ea44a98749b3cc78ad4e3a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:37.065950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e7a8a-0738-4543-bbff-fb2f8bad4794 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:37.067543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 432/870] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:36 2022] GET /identity/v3/projects/50d95139f7ea44a98749b3cc78ad4e3a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:37.073333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:37.163134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:37.168753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cbb9b-4781-4aa6-9f52-85e5cf018e42 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:37.201425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059850c7-9f45-481e-b37a-8eee276064cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:37.202061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059850c7-9f45-481e-b37a-8eee276064cf None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:37.202336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059850c7-9f45-481e-b37a-8eee276064cf None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:37.202607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059850c7-9f45-481e-b37a-8eee276064cf None admin] PATH_INFO: `/v3/projects/50d95139f7ea44a98749b3cc78ad4e3a/users/0284ef2a29064c22a1c9e5f3c318224d/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:37.204466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059850c7-9f45-481e-b37a-8eee276064cf None admin] RBAC: Authorizing `identity:create_grant(project_id=50d95139f7ea44a98749b3cc78ad4e3a, user_id=0284ef2a29064c22a1c9e5f3c318224d, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:37.205858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059850c7-9f45-481e-b37a-8eee276064cf None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:37.276582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 439/871] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:51:37 2022] PUT /identity/v3/projects/50d95139f7ea44a98749b3cc78ad4e3a/users/0284ef2a29064c22a1c9e5f3c318224d/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:51:39.375286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d2c3-97e4-4028-951d-2e392d7b6196 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:39.375815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d2c3-97e4-4028-951d-2e392d7b6196 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:39.375815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d2c3-97e4-4028-951d-2e392d7b6196 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:39.377195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 433/872] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:39.382611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:39.383045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:39.383165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:39.400737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:39.401926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:39.465894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 440/873] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:39.732714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7d2c3-97e4-4028-951d-2e392d7b6196 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:39.830077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7d2c3-97e4-4028-951d-2e392d7b6196 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:39.855659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:39.863389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:39.863714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:39.863991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:39.864289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:39.874385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:39.879460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:39.880417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:39.881357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 434/874] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:39.886687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:39.968326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:39.982254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749627e-27ef-4123-b43f-4490882824b7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:40.015747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:40.016332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:40.016612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:40.016885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:40.017543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:40.017844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:40.019791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:40.027435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 441/875] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:40.033232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:40.129413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e7bd-89dc-483b-868e-f4c2f2603dbc None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:40.161398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:40.162003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:40.162278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:40.162546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] PATH_INFO: `/v3/users/0284ef2a29064c22a1c9e5f3c318224d` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:40.163648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=0284ef2a29064c22a1c9e5f3c318224d)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:40.165078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:40.166603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 435/876] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:40 2022] GET /identity/v3/users/0284ef2a29064c22a1c9e5f3c318224d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:40.172202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:40.268614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd41a92-dfaa-4574-8fe4-bda37a935496 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:40.301138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:40.301725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:40.302000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:40.302269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:40.318825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:40.366264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:40.367680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:40.459830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 442/877] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:51:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:51:40.465648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:40.519342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:40.560859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605597d-d6e6-402b-b3da-2758a10ee3c9 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:40.592532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcafc9d-9807-4ef8-b3e3-ef50c0fa7616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:40.593139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcafc9d-9807-4ef8-b3e3-ef50c0fa7616 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:40.593418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcafc9d-9807-4ef8-b3e3-ef50c0fa7616 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:40.593688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcafc9d-9807-4ef8-b3e3-ef50c0fa7616 None admin] PATH_INFO: `/v3/projects/50d95139f7ea44a98749b3cc78ad4e3a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:40.594782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcafc9d-9807-4ef8-b3e3-ef50c0fa7616 None admin] RBAC: Authorizing `identity:get_project(project_id=50d95139f7ea44a98749b3cc78ad4e3a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:40.596211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcafc9d-9807-4ef8-b3e3-ef50c0fa7616 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:40.598344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 436/878] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:40 2022] GET /identity/v3/projects/50d95139f7ea44a98749b3cc78ad4e3a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:40.604240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:40.649873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:40.698172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4b62-895a-4e24-8447-9a7b8608586c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:40.730219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:40.730803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:40.731080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:40.731351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:40.732089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=50d95139f7ea44a98749b3cc78ad4e3a, user.id=0284ef2a29064c22a1c9e5f3c318224d) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:40.732398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:40.733790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0317723-9901-47ed-aa72-56a89f2f6d93 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:40.739332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 443/879] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:51:40 2022] GET /identity/v3/role_assignments?user.id=0284ef2a29064c22a1c9e5f3c318224d&scope.project.id=50d95139f7ea44a98749b3cc78ad4e3a&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:42.871400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb9d9-c2d9-44a1-aa00-da2b51829223 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:42.871852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb9d9-c2d9-44a1-aa00-da2b51829223 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:42.871964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb9d9-c2d9-44a1-aa00-da2b51829223 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:42.873330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 437/880] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:42.878566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19952ab-bdba-4bd4-86b7-af0bb4ff85cc None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:42.878973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19952ab-bdba-4bd4-86b7-af0bb4ff85cc None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:42.879115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19952ab-bdba-4bd4-86b7-af0bb4ff85cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:42.897275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d19952ab-bdba-4bd4-86b7-af0bb4ff85cc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:42.898453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d19952ab-bdba-4bd4-86b7-af0bb4ff85cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:42.962440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 444/881] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:43.227603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2eb9d9-c2d9-44a1-aa00-da2b51829223 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:43.327948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2eb9d9-c2d9-44a1-aa00-da2b51829223 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:43.349562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:43.359556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467e3c6-e2d6-446f-9fba-e3066269e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:43.360296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467e3c6-e2d6-446f-9fba-e3066269e036 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:43.360589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467e3c6-e2d6-446f-9fba-e3066269e036 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:43.360858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467e3c6-e2d6-446f-9fba-e3066269e036 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:43.371049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467e3c6-e2d6-446f-9fba-e3066269e036 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:43.371049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467e3c6-e2d6-446f-9fba-e3066269e036 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:43.453395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 438/882] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Oct 10 16:51:43 2022] POST /identity/v3/domains => generated 267 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:51:45.564864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd964a-17cb-4662-bf48-72a9ccc90069 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:45.565459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd964a-17cb-4662-bf48-72a9ccc90069 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:45.565459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd964a-17cb-4662-bf48-72a9ccc90069 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:45.566776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 445/883] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:45.571985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:45.572448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:45.572550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:45.590801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:45.591986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:45.655608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 439/884] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:45.922639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd964a-17cb-4662-bf48-72a9ccc90069 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:45.965577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:46.020360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd964a-17cb-4662-bf48-72a9ccc90069 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:46.052042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590c24-7536-4d2e-8ad6-eaa1257237f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:46.052673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590c24-7536-4d2e-8ad6-eaa1257237f4 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:46.052957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590c24-7536-4d2e-8ad6-eaa1257237f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:46.053234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590c24-7536-4d2e-8ad6-eaa1257237f4 None admin] PATH_INFO: `/v3/domains/4c7b99173c5d4b5190db4c98a3ffb05f` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:46.063418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76590c24-7536-4d2e-8ad6-eaa1257237f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4c7b99173c5d4b5190db4c98a3ffb05f)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:46.064866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76590c24-7536-4d2e-8ad6-eaa1257237f4 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:46.066921 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 446/885] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Oct 10 16:51:45 2022] GET /identity/v3/domains/4c7b99173c5d4b5190db4c98a3ffb05f => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:46.072629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:46.166884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7a0f-0498-43a0-893f-ae7e445c12f9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:46.180092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:46.199522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cd912-b8b1-4b14-80ff-8c5572f005ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:46.200138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd912-b8b1-4b14-80ff-8c5572f005ae None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:46.200417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd912-b8b1-4b14-80ff-8c5572f005ae None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:46.200687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd912-b8b1-4b14-80ff-8c5572f005ae None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:46.201317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cd912-b8b1-4b14-80ff-8c5572f005ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:46.202712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cd912-b8b1-4b14-80ff-8c5572f005ae None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:46.322286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 440/886] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:51:46 2022] POST /identity/v3/projects => generated 361 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:48.436816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b947f-0063-4dbf-b29a-385b662b0bce None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:48.437363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b947f-0063-4dbf-b29a-385b662b0bce None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:48.437363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b947f-0063-4dbf-b29a-385b662b0bce None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:48.438678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 447/887] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:48.443796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:48.444357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:48.444357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:48.461818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:48.462993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:48.526038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 441/888] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:48.790794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b947f-0063-4dbf-b29a-385b662b0bce None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:48.888876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b947f-0063-4dbf-b29a-385b662b0bce None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:48.890373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:48.922115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0eebc-ad5c-4fef-81c3-4d7622cd3cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:48.922734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0eebc-ad5c-4fef-81c3-4d7622cd3cda None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:48.923015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0eebc-ad5c-4fef-81c3-4d7622cd3cda None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:48.923284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0eebc-ad5c-4fef-81c3-4d7622cd3cda None admin] PATH_INFO: `/v3/domains/4c7b99173c5d4b5190db4c98a3ffb05f` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:48.924411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0eebc-ad5c-4fef-81c3-4d7622cd3cda None admin] RBAC: Authorizing `identity:get_domain(domain_id=4c7b99173c5d4b5190db4c98a3ffb05f)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:48.925810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0eebc-ad5c-4fef-81c3-4d7622cd3cda None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:48.927622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 448/889] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Oct 10 16:51:48 2022] GET /identity/v3/domains/4c7b99173c5d4b5190db4c98a3ffb05f => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:48.933681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:49.023088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:49.028621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bfb7f-60d8-4c5e-a01f-2b856bfa71bc None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:49.060782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8289a-a95a-4eba-bc83-5b5173a1af14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:49.061329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8289a-a95a-4eba-bc83-5b5173a1af14 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:49.061611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8289a-a95a-4eba-bc83-5b5173a1af14 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:49.061886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8289a-a95a-4eba-bc83-5b5173a1af14 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:49.062538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8289a-a95a-4eba-bc83-5b5173a1af14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:49.063943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8289a-a95a-4eba-bc83-5b5173a1af14 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:49.155532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 442/890] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:51:48 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:51:51.280123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64460cd2-ec9b-45e1-91c6-d6cf11c4d3a7 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:51.280674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64460cd2-ec9b-45e1-91c6-d6cf11c4d3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:51.280674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64460cd2-ec9b-45e1-91c6-d6cf11c4d3a7 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:51.281984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 449/891] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:51.291494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:51.291900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:51.292079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:51.311809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:51.313025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:51.375518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 443/892] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:51.645033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64460cd2-ec9b-45e1-91c6-d6cf11c4d3a7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:51.741374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64460cd2-ec9b-45e1-91c6-d6cf11c4d3a7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:51.836036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:51.836664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:51.836938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:51.837204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:51.847361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:51.852040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:51.853136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:51:51.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 450/893] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:51:51.859017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:51.936000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:51.953095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1ced0-0b45-4d78-b15f-a17decd180c4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:51.985404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:51.985988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:51.986260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:51.986527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:51.987210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:51.987541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:51.988927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:51.996812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 444/894] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:52.003153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:52.032810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:52.097533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd95843-a90c-449d-8258-236b618542e3 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:52.129368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:52.129972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:52.130246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:52.130512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] PATH_INFO: `/v3/users/bd9ffd3a4ba84ae3b318a9859b477381` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:52.142672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] RBAC: Authorizing `identity:get_user(user_id=bd9ffd3a4ba84ae3b318a9859b477381)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:52.144702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:52.146236 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 451/895] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:52 2022] GET /identity/v3/users/bd9ffd3a4ba84ae3b318a9859b477381 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:52.151829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:52.247416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1192e-ac11-4c83-9b58-acb9ebf8cf53 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:52.279862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3ad77-4a15-4318-8956-e42d2cf35e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:52.280512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3ad77-4a15-4318-8956-e42d2cf35e91 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:52.280792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3ad77-4a15-4318-8956-e42d2cf35e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:52.281065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3ad77-4a15-4318-8956-e42d2cf35e91 None admin] PATH_INFO: `/v3/projects/3750cb9d8c8a40e5a0d3bca641411679` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:52.282166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3ad77-4a15-4318-8956-e42d2cf35e91 None admin] RBAC: Authorizing `identity:get_project(project_id=3750cb9d8c8a40e5a0d3bca641411679)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:52.283587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3ad77-4a15-4318-8956-e42d2cf35e91 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:52.285240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 445/896] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:52 2022] GET /identity/v3/projects/3750cb9d8c8a40e5a0d3bca641411679 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:52.290988 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:52.373914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:52.385750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e31543-5ef4-4462-a299-c67cfa1c0b7a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:52.417496 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130d20d-f692-410f-968d-f7aadffa94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:52.418135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130d20d-f692-410f-968d-f7aadffa94b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:52.418409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130d20d-f692-410f-968d-f7aadffa94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:52.418675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130d20d-f692-410f-968d-f7aadffa94b5 None admin] PATH_INFO: `/v3/projects/3750cb9d8c8a40e5a0d3bca641411679/users/bd9ffd3a4ba84ae3b318a9859b477381/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:52.420546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130d20d-f692-410f-968d-f7aadffa94b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3750cb9d8c8a40e5a0d3bca641411679, user_id=bd9ffd3a4ba84ae3b318a9859b477381, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:52.421934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130d20d-f692-410f-968d-f7aadffa94b5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:52.481029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 452/897] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:51:52 2022] PUT /identity/v3/projects/3750cb9d8c8a40e5a0d3bca641411679/users/bd9ffd3a4ba84ae3b318a9859b477381/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 16:51:54.577403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee6ed9-a0b8-4f9f-ace5-8c073df23bed None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:54.577952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee6ed9-a0b8-4f9f-ace5-8c073df23bed None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:54.577952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee6ed9-a0b8-4f9f-ace5-8c073df23bed None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:54.579276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 446/898] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:51:54.584894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:54.585272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:54.585433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:54.603442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:51:54.604627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:51:54.667951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 453/899] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:51:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:51:54.937143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee6ed9-a0b8-4f9f-ace5-8c073df23bed None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:54.952765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:55.037391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee6ed9-a0b8-4f9f-ace5-8c073df23bed None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:55.070004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:55.070615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:55.070890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:55.071156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:55.080772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:51:55.085475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:51:55.086573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:51:55.087599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 447/900] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:51:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:51:55.092622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:55.165137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:55.185622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffbdda-7864-40e9-b685-1fcd9bd7dfa1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:55.219232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:55.219825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:55.220128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:55.220397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:55.221076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:55.221370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:55.222721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:55.231321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 454/901] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:51:55.237288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:55.306332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:55.341427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bc87c-34aa-4960-8b19-99a7d5e01d33 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:55.373420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:55.374037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:55.374315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:55.374585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] PATH_INFO: `/v3/users/bd9ffd3a4ba84ae3b318a9859b477381` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:55.375691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=bd9ffd3a4ba84ae3b318a9859b477381)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:55.377134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:55.379263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 448/902] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 10 16:51:55 2022] GET /identity/v3/users/bd9ffd3a4ba84ae3b318a9859b477381 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:55.385006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:55.479653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07df9f-fbe2-4210-aa39-34254a7ed843 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:55.511956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:55.512560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:55.512834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:55.513098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:55.522724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:55.576354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:55.577764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:55.668896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 455/903] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:51:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:51:55.675381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:55.770380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf973b-6456-4a80-aad7-d8e483363ff6 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:55.803302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5094d-c785-4e99-b5bf-391bc0051494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:55.803924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5094d-c785-4e99-b5bf-391bc0051494 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:55.804228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5094d-c785-4e99-b5bf-391bc0051494 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:55.804498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5094d-c785-4e99-b5bf-391bc0051494 None admin] PATH_INFO: `/v3/projects/3750cb9d8c8a40e5a0d3bca641411679` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:55.805598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5094d-c785-4e99-b5bf-391bc0051494 None admin] RBAC: Authorizing `identity:get_project(project_id=3750cb9d8c8a40e5a0d3bca641411679)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:55.807280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5094d-c785-4e99-b5bf-391bc0051494 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:55.808919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 449/904] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 10 16:51:55 2022] GET /identity/v3/projects/3750cb9d8c8a40e5a0d3bca641411679 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:51:55.814889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:51:55.853694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:51:55.908095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021214d-251d-4b76-9f82-603144caf4fd None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:51:55.940815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:51:55.941401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:51:55.941675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:51:55.941942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:51:55.942651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=3750cb9d8c8a40e5a0d3bca641411679, user.id=bd9ffd3a4ba84ae3b318a9859b477381) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:51:55.942954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:51:55.944376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b2011-a790-4303-b1aa-ab9fad5a266b None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:51:55.949810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 456/905] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:51:55 2022] GET /identity/v3/role_assignments?user.id=bd9ffd3a4ba84ae3b318a9859b477381&scope.project.id=3750cb9d8c8a40e5a0d3bca641411679&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:52:05.799661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:52:05.947326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=48 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:05.811936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6817bc-33cc-481c-ba11-3770969a015e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:05.812567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6817bc-33cc-481c-ba11-3770969a015e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:05.812567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6817bc-33cc-481c-ba11-3770969a015e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:05.813856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 450/906] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:05.818975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:05.833124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:05.833124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:05.837709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:05.838881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:05.906864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 457/907] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:06.179973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6817bc-33cc-481c-ba11-3770969a015e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:06.280739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6817bc-33cc-481c-ba11-3770969a015e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:06.312956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:06.313500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:06.313779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:06.314054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:06.324325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:06.325743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:06.340400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:06.341579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:06.342766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:06.342766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:06.342766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:06.342766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 451/908] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:06.347624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:06.443920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbbce-83fc-4df6-a447-97a7a64d1c33 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:06.471365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:06.476977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:06.477571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:06.477846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:06.478152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:06.478777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:06.479072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:06.480443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03f905-55eb-4843-881f-9cb9409c7127 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:06.491728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 458/909] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:06.497632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:06.550234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:06.594370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a8e9-2b2c-4a3e-8125-4057b9cb369e None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:06.626900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f580-8359-4d46-8bc8-6f0c9f6f5ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:06.627515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f580-8359-4d46-8bc8-6f0c9f6f5ae4 None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:06.627781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f580-8359-4d46-8bc8-6f0c9f6f5ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:06.628075 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f580-8359-4d46-8bc8-6f0c9f6f5ae4 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:06.628725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6f580-8359-4d46-8bc8-6f0c9f6f5ae4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:06.630132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6f580-8359-4d46-8bc8-6f0c9f6f5ae4 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:06.708182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 452/910] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 10 16:53:06 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:53:08.862601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a371-4b0f-4d33-8c33-ed188f71ee32 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:08.863044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a371-4b0f-4d33-8c33-ed188f71ee32 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:08.863140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a371-4b0f-4d33-8c33-ed188f71ee32 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:08.864537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 459/911] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:08.869868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:08.879936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:08.879936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:08.888609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:08.889787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:08.954289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 453/912] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:09.222959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a371-4b0f-4d33-8c33-ed188f71ee32 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:09.321836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a371-4b0f-4d33-8c33-ed188f71ee32 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:09.353674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:09.354296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:09.354567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:09.354826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:09.365048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:53:09.369688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:09.372471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:09.373529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 460/913] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:53:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:09.376859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:09.415512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:09.472716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a4c13-8838-4a3c-ad6b-8debecb50758 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:09.506532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:09.507120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:09.507399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:09.507672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:09.508363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] RBAC: Adding query filter params (name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:09.508663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:09.510017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:09.517593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 454/914] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:53:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:09.523457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:09.577842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:09.625253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62985b-f9c9-45f2-b921-f5f85c0424fc None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:09.658374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:09.658996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:09.659272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:09.659541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:09.669567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:09.671013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:09.685563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:09.686612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:09.687923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:09.687923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:09.687923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:09.687923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 461/915] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:09.693439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:09.787055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:09.789832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565eb6-cbbf-474a-bfbd-c4ec85abe65d None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:09.822777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:09.823373 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:09.823652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:09.823925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:09.824592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:09.824894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:09.826235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:09.837084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 455/916] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:09.842667 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:09.940941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c84c-f7d4-4a84-aca0-7f06c3fe4b37 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:09.942446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:09.973697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:09.974321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:09.974659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:09.974938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:09.982249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:09.983666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:53:09.995107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:09.996257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:09.997393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 462/917] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:10.002195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:10.100006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edc5a8-f476-41f9-9e51-6a1182518aa5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:10.132274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:10.132877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:10.133158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:10.133432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:10.134154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:10.134459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:10.135871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:10.152079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 456/918] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 10 16:53:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:10.157790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:10.257646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943cf1c-3439-41a1-9274-6ed3a0d6ed4c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:10.291046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:10.291666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:10.291944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:10.292245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:10.302482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:10.303922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:10.317983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:10.319326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:10.325717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:10.325717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:10.325717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:10.325717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 463/919] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:10.325993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:10.344127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:10.421344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad221ad-e4fb-4919-8b30-40d137500b6f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:10.454099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:10.454698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:10.457130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:10.457130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:10.457130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:10.457130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:10.457995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:10.468590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 457/920] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:10.474848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:10.490915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:10.571958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcccb99-1737-444c-b6c3-c55035c872c9 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:10.604949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:10.605565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:10.605839 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:10.606100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:10.612466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:10.613895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:53:10.624712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:10.625811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:10.626941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 464/921] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:53:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:53:10.632233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:10.711022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:10.731381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33aeced-210b-41ab-bd19-4d402b4fe54b None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:10.764415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:10.765015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:10.765296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:10.765568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:10.766250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:10.766551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:10.767962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd03b5-426a-434d-bb57-c890ca1f663d None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:10.781104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 458/922] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:53:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:10.787201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:10.860458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:10.884867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb45b-4f9c-44b1-8fe7-b2488dee801a None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:10.917333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d790a19-4776-4e1e-80e6-fee7e17f03aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:10.917975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790a19-4776-4e1e-80e6-fee7e17f03aa None admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:10.918273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790a19-4776-4e1e-80e6-fee7e17f03aa None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:10.918539 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d790a19-4776-4e1e-80e6-fee7e17f03aa None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/d882630eb293423e84ecbd0c63fb6733/roles/b57b1b5ed1d94183bf8e41ba05644b01` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:10.932295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d790a19-4776-4e1e-80e6-fee7e17f03aa None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=d882630eb293423e84ecbd0c63fb6733, role_id=b57b1b5ed1d94183bf8e41ba05644b01)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:10.938356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d790a19-4776-4e1e-80e6-fee7e17f03aa None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:11.016461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 465/923] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:53:10 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/d882630eb293423e84ecbd0c63fb6733/roles/b57b1b5ed1d94183bf8e41ba05644b01 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:53:13.132698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86156-265b-466c-8192-b68838675c9e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:13.133216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86156-265b-466c-8192-b68838675c9e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:13.133216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86156-265b-466c-8192-b68838675c9e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:13.134535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 459/924] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:13.139671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:13.140119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:13.140232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:13.157656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:13.162553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:13.227363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 466/925] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:13.496752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de86156-265b-466c-8192-b68838675c9e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:13.596069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de86156-265b-466c-8192-b68838675c9e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:13.628222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:13.629361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:13.629631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:13.629894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:13.636411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:13.640065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:53:13.645299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:13.650915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:13.651928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 460/926] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 10 16:53:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:13.652653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:13.712171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:13.754188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b27c4f-7d2f-463b-ba8e-e47de1bb11d7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:13.786485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:13.787086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:13.787365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:13.787637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:13.788319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:13.788620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:13.789970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:13.797548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 467/927] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 10 16:53:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:53:13.803542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:13.901835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9e56e-fbd0-4c85-b9dc-b21c40b05d45 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:13.933930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:13.934541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:13.934813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:13.935078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:13.945184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:13.946638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:13.961361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:13.962385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:13.963716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:13.963716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:13.963716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:13.963716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 461/928] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:13.968229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:14.064066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35e06-9eda-43b2-b1fd-f771f0c7b7f7 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:14.079769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:14.097083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:14.097815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:14.098097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:14.098372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:14.099017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:14.099321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:14.100732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:14.112175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 468/929] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:14.118490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:14.167962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:14.215450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c6f79-42a5-4fba-bf8f-77215b225371 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:14.247753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:14.248392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:14.248671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:14.248930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:14.256420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:14.257908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:53:14.270512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:14.272465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:14.273389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 462/930] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:14.277626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:14.374418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60fe40c-535d-4bf2-aa2e-469f93c3c15d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:14.406145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:14.406745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:14.407019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:14.407286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:14.407994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:14.408324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:14.410228 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:14.427133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 469/931] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 10 16:53:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:14.432893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:14.528569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded3bd3-4df6-47b3-9cd8-b607ca031215 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:14.539003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:14.561741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:14.562339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:14.562615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:14.562883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:14.627411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:14.628842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:14.725269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 463/932] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:53:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:53:14.731216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:14.766893 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:14.829184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378186d-a599-4d49-af04-9accfdb9b206 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:14.861619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:14.862247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:14.862528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:14.862824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:14.872834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:14.874257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:14.888483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:14.889659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:14.890775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:14.890775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:14.890775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:14.890775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 470/933] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:14.895824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:14.907041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:14.990404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cab88-d198-4eb3-b6f1-69f8f4f7b98f None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:15.022464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:15.023047 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:15.023319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:15.023587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:15.024241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:15.024539 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:15.025867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:15.037010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 464/934] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:15.042737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:15.126235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:15.140870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457db95-5815-4885-85eb-31ad527cc648 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:15.172541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:15.173145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:15.173417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:15.173682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:15.179945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:15.181645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:53:15.193663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:15.198140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:15.199246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 471/935] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:53:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:15.200941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:15.268017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:15.298701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17071c2-ef65-4746-9fe6-4b882c18dbd5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:15.330649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:15.331240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:15.331514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:15.331779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:15.333136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:15.333437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:15.334880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24ebbc-2b66-4dba-b75e-8169781c1a05 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:15.347278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 465/936] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:53:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:15.353195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:15.450536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204057ca-2932-48ce-9172-01c0aeda2a82 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:15.484174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:15.484764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:15.485044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:15.485308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:15.486015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] RBAC: Adding query filter params (role.id=b57b1b5ed1d94183bf8e41ba05644b01, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=d882630eb293423e84ecbd0c63fb6733) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:15.486316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:15.487705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c61d0-b1b4-4842-b488-d4d2e5127da3 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:15.493890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 472/937] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:53:15 2022] GET /identity/v3/role_assignments?user.id=d882630eb293423e84ecbd0c63fb6733&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=b57b1b5ed1d94183bf8e41ba05644b01 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:17.652442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f514d7-ae44-4db0-b04a-c8769b581b55 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:17.653018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f514d7-ae44-4db0-b04a-c8769b581b55 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:17.653018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f514d7-ae44-4db0-b04a-c8769b581b55 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:17.654306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 466/938] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:17.659446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:17.659900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:17.660002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:17.679522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:17.680978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:17.683644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:17.748731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 473/939] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:18.012594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f514d7-ae44-4db0-b04a-c8769b581b55 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:18.111299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f514d7-ae44-4db0-b04a-c8769b581b55 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:18.139261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:18.145492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:18.146108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:18.146380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:18.146687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:18.156450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:53:18.161194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:18.163133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:18.164128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 467/940] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:18.168542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:18.264535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f1bb-6ddb-4f82-9391-658bda41519b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:18.297910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:18.298510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:18.298788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:18.299058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:18.299712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:18.300012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:18.301397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:18.309394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 474/941] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:53:18.315212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:18.411842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbada72-abd3-444d-b1a6-eacdfd9db651 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:18.444182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:18.444793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:18.445066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:18.445334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:18.454829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:18.456276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:18.470850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:18.475947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:18.477038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:18.477038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:18.477038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:18.477038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 468/942] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:18.477880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:18.480774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:18.577476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff1150-fa18-459d-9059-20edfab26e6d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:18.610624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:18.611217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:18.611492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:18.611758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:18.612412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:18.612711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:18.614125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:18.626592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 475/943] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:18.632384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:18.675432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:18.729822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39f27-794e-4abc-b95e-7784fd2f6e15 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:18.762749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:18.763367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:18.763642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:18.763915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:18.771489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:18.772923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:53:18.783645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:18.790872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:18.791820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 469/944] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:18.792514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:18.855116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:18.892422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e96e3-955b-4c84-8a2f-1baac3be4aac None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:18.924846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:18.925449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:18.925760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:18.926035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:18.926754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:18.927059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:18.928503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:18.946166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 476/945] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:18.952002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:19.050011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d212d0e-0ac1-4ca4-b296-b8f16bd0e7f3 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:19.051566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:19.084335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:19.084981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:19.085259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:19.085530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:19.094967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:19.096453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:19.113646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:19.114777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:19.115884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:19.115884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:19.115884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:19.115884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 470/946] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:19.117546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:19.213713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23b80d-c0d8-4e11-9eac-bbe88f1dc992 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:19.232649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:19.246309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:19.246931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:19.247178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:19.247443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:19.248136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:19.248435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:19.249812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:19.261083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 477/947] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:19.266721 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:19.366089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6224d0d-b198-4834-a52e-dc21d91c830a None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:19.398569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:19.399191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:19.399470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:19.399740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:19.406203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:19.407668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:53:19.422122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:19.423248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:19.424233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 471/948] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:19.429855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:19.528365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94bc3-9482-44a6-844f-155ed79afcb1 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:19.561654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:19.562252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:19.562529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:19.562798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:19.563479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:19.563777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:19.565238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5485e0-5ae1-422e-8106-a0aaabcbe8de None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:19.577419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 478/949] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:53:19.584208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:19.604035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:19.685036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5c701-d0e4-489d-b2b5-a2e7cd0840a1 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:19.718360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f57cb-5313-4d27-8a08-fd598dfe9e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:19.719012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f57cb-5313-4d27-8a08-fd598dfe9e91 None admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:19.719288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f57cb-5313-4d27-8a08-fd598dfe9e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:19.719557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f57cb-5313-4d27-8a08-fd598dfe9e91 None admin] PATH_INFO: `/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/d882630eb293423e84ecbd0c63fb6733/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:19.721578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f57cb-5313-4d27-8a08-fd598dfe9e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=450c1373880a4e2ba0eb9fd1c5201580, user_id=d882630eb293423e84ecbd0c63fb6733, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:19.722981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f57cb-5313-4d27-8a08-fd598dfe9e91 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:19.828591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 472/950] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 10 16:53:19 2022] PUT /identity/v3/projects/450c1373880a4e2ba0eb9fd1c5201580/users/d882630eb293423e84ecbd0c63fb6733/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 16:53:21.974304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabffdd-7d14-4f7d-bb80-2b4ae72248b3 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:21.974991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabffdd-7d14-4f7d-bb80-2b4ae72248b3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:21.974991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabffdd-7d14-4f7d-bb80-2b4ae72248b3 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:21.976205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 479/951] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:21.981336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:21.981775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:21.981867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:21.999512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:22.000716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:22.065609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 473/952] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:22.338897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabffdd-7d14-4f7d-bb80-2b4ae72248b3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:22.383565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:22.441039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabffdd-7d14-4f7d-bb80-2b4ae72248b3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:22.474324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:22.474943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:22.475218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:22.475486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:22.485335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 16:53:22.490683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:22.491870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:22.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 480/953] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 10 16:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:22.498199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:22.501381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:22.594388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e3eea-ea4a-496e-9889-b10adda0759e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:22.629235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:22.629830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:22.630107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:22.630375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:22.631097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:22.631396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:22.632764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:22.640603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 474/954] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 10 16:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:22.646474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:22.742149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:22.745055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78018-6415-4a78-a653-17afe9ea649c None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:22.777090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:22.777708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:22.777985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:22.778258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:22.788408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:22.789830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:22.804535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:22.805664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:22.807111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:22.807111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:22.807111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:22.807111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 481/955] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:22.811630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:22.875469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:22.908896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba453d6f-774b-414e-acb6-d43b292967e8 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:22.941226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:22.941825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:22.942099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:22.942363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:22.942990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:22.943286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:22.944661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:22.957912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 475/956] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:22.962830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:23.060391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31af6ae-3626-4c43-918c-6f4949bc8a20 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:23.092891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:23.093514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:23.093791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:23.094062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:23.101740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:23.103153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 16:53:23.113905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:23.114980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:23.116326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 482/957] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:23.121008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:23.219933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab242f-cd5e-4077-a981-ede8db5ed022 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:23.244553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:23.253095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:23.257110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:23.257110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:23.257110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:23.257110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:23.257110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:23.257110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:23.274711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 476/958] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 10 16:53:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:23.280680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:23.296725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:23.379920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ca4a5-07a2-4384-a983-f4cea7fbaafb None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:23.413841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:23.417280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:23.417280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:23.417280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:23.480819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:23.482304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:23.496517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:23.578380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 483/959] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 10 16:53:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 16:53:23.584710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:23.685290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c253cd-575c-4354-8823-c7d266586167 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:23.719205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:23.719827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:23.720130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:23.720403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:23.730289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:23.731811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 16:53:23.747462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:23.751677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 16:53:23.752812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 16:53:23.752812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 16:53:23.752812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:23.752812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 477/960] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 16:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 16:53:23.755131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:23.850974 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e829e8-8c87-4c31-ae30-b656c090703d None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:23.864634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:23.885568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:23.886167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:23.886440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:23.886708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:23.887338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:23.887636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:23.889024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:23.900122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 484/961] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 10 16:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:23.905937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:23.940957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:24.003175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991a404-eabf-49cc-a6f9-01b29acb3da5 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:24.038401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:24.039022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:24.039296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:24.039560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:24.045957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:24.047356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 16:53:24.059132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:24.061924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: ERROR keystone.server.flask.application Oct 10 16:53:24.063032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 478/962] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 10 16:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:24.067173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:24.162654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d46e3c-1cde-40f8-91fe-80fde1811995 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:24.195165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:24.195713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:24.195996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:24.196304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:24.196989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:24.197294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:24.198719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc79549-0043-49d0-b944-d8cc0c948a9d None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:24.211822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 485/963] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 10 16:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:24.218128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:24.313356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:24.319069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bd676-87f1-4a94-ac75-9ce9b73b9915 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:24.351326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:24.351918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:24.352324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:24.358731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:24.358731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] RBAC: Adding query filter params (role.id=6e700d1e090747d6acdf236f6761cbea, scope.project.id=450c1373880a4e2ba0eb9fd1c5201580, user.id=d882630eb293423e84ecbd0c63fb6733) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:24.358731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:24.358731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdd9b3-860b-4a3f-9381-a1cfea3891a6 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:24.360830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 479/964] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 10 16:53:24 2022] GET /identity/v3/role_assignments?user.id=d882630eb293423e84ecbd0c63fb6733&scope.project.id=450c1373880a4e2ba0eb9fd1c5201580&role.id=6e700d1e090747d6acdf236f6761cbea => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:26.487328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8f1c4-9b5a-4527-b9b2-0cf4458660de None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:26.487887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8f1c4-9b5a-4527-b9b2-0cf4458660de None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:26.488553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8f1c4-9b5a-4527-b9b2-0cf4458660de None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:26.489884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 486/965] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:26.495213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:26.495649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:26.495760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:26.513369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:26.514534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:26.578485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 480/966] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:26.841877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8f1c4-9b5a-4527-b9b2-0cf4458660de None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:26.873527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:26.942440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8f1c4-9b5a-4527-b9b2-0cf4458660de None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:26.976003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:26.976650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:26.976929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:26.977197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:26.977772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:26.979117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:26.987625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:26.994273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:26.995310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 487/967] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:53:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:26.996421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:27.081431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:27.091572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58fdcf-ec79-43ea-bb47-d6cf875dae31 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:27.130010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:27.130650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:27.130971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:27.131238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:27.131862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:27.132325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:27.133536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3878a-bb3b-4458-87d0-b06d72801a60 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:27.138374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 481/968] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 10 16:53:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:27.145002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:27.244841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:27.250500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21977a47-1aa1-4dd7-b152-d840851da7b8 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:27.284652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:27.285302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:27.285584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:27.285857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] PATH_INFO: `/v3/services` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:27.286492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:27.286798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:27.288180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e0576-232b-4eb5-8704-656831bc8ba5 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:27.294820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 488/969] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 10 16:53:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 16:53:29.427594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df6c71-46f1-4790-8496-8f0435281468 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:29.428169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df6c71-46f1-4790-8496-8f0435281468 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:29.428169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df6c71-46f1-4790-8496-8f0435281468 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:29.429521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 482/970] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 16:53:29.435445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3365aec-2b54-45aa-bea7-a0df97116e1c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:29.438211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3365aec-2b54-45aa-bea7-a0df97116e1c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:29.438211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3365aec-2b54-45aa-bea7-a0df97116e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:29.453681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a3365aec-2b54-45aa-bea7-a0df97116e1c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:29.454860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a3365aec-2b54-45aa-bea7-a0df97116e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:29.521808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 489/971] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:29.798324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df6c71-46f1-4790-8496-8f0435281468 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:29.903842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df6c71-46f1-4790-8496-8f0435281468 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:29.937991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e0b76-3cf3-4019-89cd-db267811e3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:29.938611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e0b76-3cf3-4019-89cd-db267811e3af None admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:29.938892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e0b76-3cf3-4019-89cd-db267811e3af None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:29.939162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e0b76-3cf3-4019-89cd-db267811e3af None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:29.939830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e0b76-3cf3-4019-89cd-db267811e3af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:29.941247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e0b76-3cf3-4019-89cd-db267811e3af None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:30.017088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 483/972] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 10 16:53:29 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 16:53:32.182067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c24b-7262-4821-8b17-50da7da95796 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:32.182599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c24b-7262-4821-8b17-50da7da95796 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:32.182599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c24b-7262-4821-8b17-50da7da95796 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:32.183956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 490/973] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:32.189097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:32.189514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:32.189650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:32.208012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:32.209213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:32.218002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:32.286193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 484/974] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:32.560018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c24b-7262-4821-8b17-50da7da95796 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:32.653514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:32.663911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c24b-7262-4821-8b17-50da7da95796 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:32.696924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:32.697550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:32.697830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:32.698099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] PATH_INFO: `/v3/services/placement` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:32.698678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:32.700057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:32.709567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:32.711833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:32.712909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 491/975] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:53:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:32.716802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:32.811514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f2d50-69ec-45c3-a9ac-5d787c2f8031 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:32.844047 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:32.844669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:32.844948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:32.845220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:32.845842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:32.846150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:32.848283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:32.853350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 485/976] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 10 16:53:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:32.861602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:32.958237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b41d2-ec2a-4eb4-84a9-5d24c3b8d8ed None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:32.990104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:32.990693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:32.990969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:32.991234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:32.991914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] RBAC: Adding query filter params (interface=public, service_id=63a2d58ae38f404a97a552ff87cd7e73, region_id=RegionOne) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:32.992249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:32.993587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8be35-21a3-4a27-86d1-b4de37fedd92 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:32.998329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 492/977] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 10 16:53:32 2022] GET /identity/v3/endpoints?service_id=63a2d58ae38f404a97a552ff87cd7e73&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:33.004336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:33.008008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:33.100799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e6d73-66a3-4552-9c4f-71f1210eab27 None admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:33.132744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d63ecd-8092-468f-b43b-886d813cb309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:33.133335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63ecd-8092-468f-b43b-886d813cb309 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:33.133613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63ecd-8092-468f-b43b-886d813cb309 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:33.133883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63ecd-8092-468f-b43b-886d813cb309 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:33.134486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d63ecd-8092-468f-b43b-886d813cb309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:33.135863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d63ecd-8092-468f-b43b-886d813cb309 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:33.142418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 486/978] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 10 16:53:33 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 16:53:35.264467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7319ed9-9092-40fc-8588-87bc52b8ff05 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:35.265023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7319ed9-9092-40fc-8588-87bc52b8ff05 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:35.265023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7319ed9-9092-40fc-8588-87bc52b8ff05 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:35.266483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 493/979] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:53:35.271864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:35.272288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:35.272523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:35.290160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:53:35.291330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:53:35.352092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:35.357485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 487/980] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:53:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 16:53:35.626272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7319ed9-9092-40fc-8588-87bc52b8ff05 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:35.646593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:35.727955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7319ed9-9092-40fc-8588-87bc52b8ff05 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:35.759676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:35.760320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:35.760595 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:35.760861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:35.761434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:35.762778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 16:53:35.771857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 16:53:35.773137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application with Lock( Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self._enter() Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application return self.creator() Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application created_value = creator( Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: ERROR keystone.server.flask.application Oct 10 16:53:35.774137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 494/981] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 10 16:53:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 16:53:35.778929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:35.877850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d75c8-21d3-4161-b28a-2594d76d6cd9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:35.911390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:35.911995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:35.912308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:35.912579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] PATH_INFO: `/v3/services` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:35.913201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 16:53:35.913501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:35.914876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771aed8d-c22b-43dc-93d2-17d121e06cd3 None admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:35.919832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 488/982] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 10 16:53:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 16:53:35.925727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:53:36.004375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:36.022425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc62fa-ad94-4077-9efc-9c123c680518 None admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:53:36.054356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d8017-139e-4999-804d-55e64679c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:53:36.054967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d8017-139e-4999-804d-55e64679c073 None admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:53:36.055240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d8017-139e-4999-804d-55e64679c073 None admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:53:36.055505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d8017-139e-4999-804d-55e64679c073 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:53:36.056457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d8017-139e-4999-804d-55e64679c073 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:53:36.057800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d8017-139e-4999-804d-55e64679c073 None admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:53:36.147999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 495/983] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 16:53:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 16:53:45.916879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=69 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:53:46.142615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:38.174988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4acc49-ac54-40c1-a06c-f3f786356b7b None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:38.175529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4acc49-ac54-40c1-a06c-f3f786356b7b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:38.175529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4acc49-ac54-40c1-a06c-f3f786356b7b None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:38.182304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f3fd6-edfc-4d4e-a943-054c5c4e2581 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:38.182788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f3fd6-edfc-4d4e-a943-054c5c4e2581 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:38.182871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f3fd6-edfc-4d4e-a943-054c5c4e2581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:38.187606 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 489/984] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:38.260508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-721f3fd6-edfc-4d4e-a943-054c5c4e2581 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:38.261782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-721f3fd6-edfc-4d4e-a943-054c5c4e2581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:38.366926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 496/985] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:38.656084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05750e-a100-469e-86a2-bec79d4695f4 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:38.656685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05750e-a100-469e-86a2-bec79d4695f4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:38.656961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05750e-a100-469e-86a2-bec79d4695f4 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:38.658520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 490/986] 10.222.0.42 () {54 vars in 964 bytes} [Mon Oct 10 16:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:38.664508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a8da9-49ba-447b-9622-8b04ede8fe0e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:38.664798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a8da9-49ba-447b-9622-8b04ede8fe0e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:38.665072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a8da9-49ba-447b-9622-8b04ede8fe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:38.745783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4c9a8da9-49ba-447b-9622-8b04ede8fe0e None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:38.746972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4c9a8da9-49ba-447b-9622-8b04ede8fe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:38.770386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:38.836467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 497/987] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Oct 10 16:59:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:38.845765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05750e-a100-469e-86a2-bec79d4695f4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:38.847519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ba05750e-a100-469e-86a2-bec79d4695f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:38.973902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05750e-a100-469e-86a2-bec79d4695f4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:39.059209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:39.059822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:39.060121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:39.060395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:39.062371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:39.122813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:39.181819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:39.184017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3664b-b0f4-40d2-b533-fd7c1036042b service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:39.204645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 491/988] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 16:59:41.635246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32023e7-2001-4da1-84f7-597a962b5b48 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:41.635755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32023e7-2001-4da1-84f7-597a962b5b48 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:41.635848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32023e7-2001-4da1-84f7-597a962b5b48 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:41.637195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 498/989] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:41.642445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a04491-90cd-4f55-b6c4-733bb3f95fa7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:41.642845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a04491-90cd-4f55-b6c4-733bb3f95fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:41.643112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a04491-90cd-4f55-b6c4-733bb3f95fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:41.724450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-95a04491-90cd-4f55-b6c4-733bb3f95fa7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:41.725655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-95a04491-90cd-4f55-b6c4-733bb3f95fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:41.818774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 492/990] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:42.112367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f9c9c-e4b6-4a78-9472-1b3af6e221fd None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:42.112654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f9c9c-e4b6-4a78-9472-1b3af6e221fd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:42.112935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f9c9c-e4b6-4a78-9472-1b3af6e221fd None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:42.114285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 499/991] 10.222.0.42 () {54 vars in 964 bytes} [Mon Oct 10 16:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:42.120297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bbe55-f0bc-4082-afa0-4f00e0a32840 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:42.120584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bbe55-f0bc-4082-afa0-4f00e0a32840 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:42.120860 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bbe55-f0bc-4082-afa0-4f00e0a32840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:42.133843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:42.197412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-dc2bbe55-f0bc-4082-afa0-4f00e0a32840 None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:42.198586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dc2bbe55-f0bc-4082-afa0-4f00e0a32840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:42.274082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 493/992] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Oct 10 16:59:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:42.283086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f9c9c-e4b6-4a78-9472-1b3af6e221fd None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:42.284696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-533f9c9c-e4b6-4a78-9472-1b3af6e221fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:42.394851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f9c9c-e4b6-4a78-9472-1b3af6e221fd None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:42.470946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:42.473761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:42.474390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:42.474673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:42.474947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:42.476636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:42.588687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:42.590110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afbd6a-5705-48a7-b0cc-1a97cb493b3e service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:42.609774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 500/993] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 16:59:45.434498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020f2c0-97a1-4c47-996e-672e18d08caf None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:45.435050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020f2c0-97a1-4c47-996e-672e18d08caf None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:45.435169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020f2c0-97a1-4c47-996e-672e18d08caf None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:45.436453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 494/994] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:45.442161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60dd8-b070-4112-8f31-e346f01fd65f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:45.442543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60dd8-b070-4112-8f31-e346f01fd65f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:45.442710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60dd8-b070-4112-8f31-e346f01fd65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:45.507180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-cda60dd8-b070-4112-8f31-e346f01fd65f None None] MFA Rules not processed for user `54ccb7a8df554b2ca4677075285913c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:45.508402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cda60dd8-b070-4112-8f31-e346f01fd65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:45.559805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:45.603891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 501/995] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:45.885208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020f2c0-97a1-4c47-996e-672e18d08caf None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:45.886763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d020f2c0-97a1-4c47-996e-672e18d08caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:45.986566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020f2c0-97a1-4c47-996e-672e18d08caf None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:45.989312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:46.057902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:46.058497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:46.058771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:46.059036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:46.060617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:46.164580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:46.165996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132f824-8c76-4c26-bc3e-50d2ea1281ac service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:46.184546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 495/996] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 16:59:49.103666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c905-b0e4-4aee-bb44-be343ff2de5f None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:49.104251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c905-b0e4-4aee-bb44-be343ff2de5f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:49.104251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c905-b0e4-4aee-bb44-be343ff2de5f None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:49.105613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 502/997] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:49.110853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f55911-58e3-4efd-af77-1c2fbee14e26 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:49.111285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f55911-58e3-4efd-af77-1c2fbee14e26 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:49.111414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f55911-58e3-4efd-af77-1c2fbee14e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:49.151714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:49.176856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-62f55911-58e3-4efd-af77-1c2fbee14e26 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:49.178055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-62f55911-58e3-4efd-af77-1c2fbee14e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:49.271495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 496/998] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 16:59:49.557465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9c905-b0e4-4aee-bb44-be343ff2de5f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:49.558975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2fb9c905-b0e4-4aee-bb44-be343ff2de5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:49.657609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9c905-b0e4-4aee-bb44-be343ff2de5f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:49.728144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:49.728736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:49.729010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:49.729278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:49.730835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:49.738148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:49.835743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:49.837119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e6b6e-0220-4dd5-8438-f5bb8f204e8e service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:49.854678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 503/999] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 16:59:52.979808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133148b6-682f-41de-91b5-99e0501731a4 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:52.980392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133148b6-682f-41de-91b5-99e0501731a4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:52.980553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133148b6-682f-41de-91b5-99e0501731a4 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:52.982016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 497/1000] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:52.987411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddec57-21bc-49bf-b63a-4b4712be7836 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:52.987844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddec57-21bc-49bf-b63a-4b4712be7836 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:52.987967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddec57-21bc-49bf-b63a-4b4712be7836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:53.052588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f6ddec57-21bc-49bf-b63a-4b4712be7836 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:53.053784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f6ddec57-21bc-49bf-b63a-4b4712be7836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:53.116109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:53.163211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 504/1001] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:53.443477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133148b6-682f-41de-91b5-99e0501731a4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:53.445054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-133148b6-682f-41de-91b5-99e0501731a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:53.546498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133148b6-682f-41de-91b5-99e0501731a4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:53.561000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:53.621334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:53.621934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:53.622207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:53.622474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:53.624044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:53.730529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:53.731868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fc085-51e8-45f7-ac41-38601087b1ce service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:53.750992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 498/1002] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 16:59:55.969108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f69fc-ba4d-42ef-be3e-ad88ed1ee7b0 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:55.969722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f69fc-ba4d-42ef-be3e-ad88ed1ee7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:55.969722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f69fc-ba4d-42ef-be3e-ad88ed1ee7b0 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:55.971560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 505/1003] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:55.977291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622c0d8-90a7-466b-bfb1-05e1744092f5 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:55.977756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622c0d8-90a7-466b-bfb1-05e1744092f5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:55.977857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622c0d8-90a7-466b-bfb1-05e1744092f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:56.026850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:56.045170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d622c0d8-90a7-466b-bfb1-05e1744092f5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:56.046385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d622c0d8-90a7-466b-bfb1-05e1744092f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:56.148068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 499/1004] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:56.431249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f69fc-ba4d-42ef-be3e-ad88ed1ee7b0 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:56.432255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-663f69fc-ba4d-42ef-be3e-ad88ed1ee7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:56.537947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f69fc-ba4d-42ef-be3e-ad88ed1ee7b0 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:56.609938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:56.610546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:56.610826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:56.611096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:56.612720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:56.629863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:56.724669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:56.726035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31ee11-0378-4c7d-b080-6b2533484e56 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:56.743677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 506/1005] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 16:59:58.963764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d26bd3-2630-4637-949f-87c30cb853ca None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:58.964300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d26bd3-2630-4637-949f-87c30cb853ca None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:58.964300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d26bd3-2630-4637-949f-87c30cb853ca None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:58.965652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 500/1006] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 16:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 16:59:58.971158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad38ee0-f54f-40d1-9fc3-1e64f75e39ca None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:58.971604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad38ee0-f54f-40d1-9fc3-1e64f75e39ca None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:58.971705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad38ee0-f54f-40d1-9fc3-1e64f75e39ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:59.036576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9ad38ee0-f54f-40d1-9fc3-1e64f75e39ca None None] MFA Rules not processed for user `54ccb7a8df554b2ca4677075285913c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 16:59:59.037761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9ad38ee0-f54f-40d1-9fc3-1e64f75e39ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:59.102460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:59.133666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 507/1007] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 16:59:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 16:59:59.419074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d26bd3-2630-4637-949f-87c30cb853ca None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 16:59:59.420635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f9d26bd3-2630-4637-949f-87c30cb853ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:59.526744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d26bd3-2630-4637-949f-87c30cb853ca None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 16:59:59.547592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 16:59:59.606855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 16:59:59.607461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 16:59:59.607733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 16:59:59.607996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 16:59:59.609625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 16:59:59.725438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 16:59:59.726928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4c4c8-f1d2-4dbd-a23f-4d32f384336d service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 16:59:59.747440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 501/1008] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 16:59:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:02.274914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ddfa2-90f6-4ce9-92ba-ab9ee1591b3e None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:02.275476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ddfa2-90f6-4ce9-92ba-ab9ee1591b3e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:02.275476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ddfa2-90f6-4ce9-92ba-ab9ee1591b3e None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:02.276872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 508/1009] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:02.282039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811d51-06b9-4bb5-b40e-f4b4c9bf62c0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:02.282475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811d51-06b9-4bb5-b40e-f4b4c9bf62c0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:02.283106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811d51-06b9-4bb5-b40e-f4b4c9bf62c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:02.346462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:02.349291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-89811d51-06b9-4bb5-b40e-f4b4c9bf62c0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:02.350490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-89811d51-06b9-4bb5-b40e-f4b4c9bf62c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:02.444580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 502/1010] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:02.722891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ddfa2-90f6-4ce9-92ba-ab9ee1591b3e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:02.724467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f71ddfa2-90f6-4ce9-92ba-ab9ee1591b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:02.838198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ddfa2-90f6-4ce9-92ba-ab9ee1591b3e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:02.910570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:02.911175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:02.911452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:02.911724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:02.913329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:02.940912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:03.020569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:03.021922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69dec4-12f9-4ed2-8a17-627429286971 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:03.039307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 509/1011] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:05.218410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f605d-95d9-4db3-9710-c32779b7a92b None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:05.218972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f605d-95d9-4db3-9710-c32779b7a92b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:05.218972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f605d-95d9-4db3-9710-c32779b7a92b None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:05.220370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 503/1012] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:05.226484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec807f-3bfe-45a5-82fe-6ea169b64154 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:05.226907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec807f-3bfe-45a5-82fe-6ea169b64154 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:05.227036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec807f-3bfe-45a5-82fe-6ea169b64154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:05.291598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-54ec807f-3bfe-45a5-82fe-6ea169b64154 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:05.292832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-54ec807f-3bfe-45a5-82fe-6ea169b64154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:05.365933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:05.388004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 510/1013] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:05.681700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f605d-95d9-4db3-9710-c32779b7a92b None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:05.683222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-114f605d-95d9-4db3-9710-c32779b7a92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:05.783041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f605d-95d9-4db3-9710-c32779b7a92b None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:05.810023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:05.854747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:05.855283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:05.855561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:05.855833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:05.857441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:05.964474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:05.965892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4a076-b2ad-4a25-9a9e-d7d76917ab70 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:05.984922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 504/1014] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:08.808247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893750d-0f4b-4adb-bd6e-db3ec46f8391 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:08.808787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893750d-0f4b-4adb-bd6e-db3ec46f8391 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:08.808787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893750d-0f4b-4adb-bd6e-db3ec46f8391 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:08.810132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 511/1015] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:08.816151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31632a5b-17cc-4fdc-89ae-932e5cac472a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:08.816579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31632a5b-17cc-4fdc-89ae-932e5cac472a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:08.816698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31632a5b-17cc-4fdc-89ae-932e5cac472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:08.885665 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-31632a5b-17cc-4fdc-89ae-932e5cac472a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:08.886885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-31632a5b-17cc-4fdc-89ae-932e5cac472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:08.895531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:08.981370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 505/1016] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:09.264731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893750d-0f4b-4adb-bd6e-db3ec46f8391 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:09.266252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a893750d-0f4b-4adb-bd6e-db3ec46f8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:09.366754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893750d-0f4b-4adb-bd6e-db3ec46f8391 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:09.439322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:09.439923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:09.440229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:09.440500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:09.442081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:09.476304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:09.547441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:09.549399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896fb31-8946-4d18-9368-ee27f2940629 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:09.567556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 512/1017] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:13.814244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0dc41-79e7-4fea-8b18-3a354dfc6e6a None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:13.814244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0dc41-79e7-4fea-8b18-3a354dfc6e6a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:13.814244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0dc41-79e7-4fea-8b18-3a354dfc6e6a None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:13.814244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 506/1018] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:13.814870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e099c-78ba-480a-9f94-7512b9ac4a0e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:13.815217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e099c-78ba-480a-9f94-7512b9ac4a0e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:13.815315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e099c-78ba-480a-9f94-7512b9ac4a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:13.881431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9c6e099c-78ba-480a-9f94-7512b9ac4a0e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:13.882630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9c6e099c-78ba-480a-9f94-7512b9ac4a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:13.967459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:13.978204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 513/1019] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:14.262361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0dc41-79e7-4fea-8b18-3a354dfc6e6a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:14.263892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-17f0dc41-79e7-4fea-8b18-3a354dfc6e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:14.367597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0dc41-79e7-4fea-8b18-3a354dfc6e6a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:14.405475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:14.442767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:14.443374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:14.443655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:14.443926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:14.445547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:14.554210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:14.555543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb13a9-79f0-45cc-881e-63123576fe63 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:14.575226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 507/1020] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:17.713463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb1392-ba57-40d3-bbd2-54c7d13dfcf8 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:17.713966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb1392-ba57-40d3-bbd2-54c7d13dfcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:17.714079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb1392-ba57-40d3-bbd2-54c7d13dfcf8 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:17.715365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 514/1021] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:17.720561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba3947-de50-4845-9b42-da465e5200c0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:17.720983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba3947-de50-4845-9b42-da465e5200c0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:17.721122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba3947-de50-4845-9b42-da465e5200c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:17.793962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-dbba3947-de50-4845-9b42-da465e5200c0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:17.795178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dbba3947-de50-4845-9b42-da465e5200c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:17.813023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:17.894486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 508/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:18.181554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb1392-ba57-40d3-bbd2-54c7d13dfcf8 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:18.183194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-31eb1392-ba57-40d3-bbd2-54c7d13dfcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:18.287474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb1392-ba57-40d3-bbd2-54c7d13dfcf8 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:18.363275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:18.363892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:18.364214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:18.364476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:18.366063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:18.408865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:18.488579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:18.490017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ad5e-cc0e-4f45-a31c-8025f4d28970 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:18.513732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 515/1023] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:22.989189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc788a-3e02-4b86-a36b-5aa5baa60d32 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:22.989705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc788a-3e02-4b86-a36b-5aa5baa60d32 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:22.989794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc788a-3e02-4b86-a36b-5aa5baa60d32 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:22.991121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 509/1024] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:22.996683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77e79-a695-4433-808d-eec24f33c963 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:22.997113 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77e79-a695-4433-808d-eec24f33c963 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:22.997226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77e79-a695-4433-808d-eec24f33c963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:23.064516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-fae77e79-a695-4433-808d-eec24f33c963 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:23.065725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fae77e79-a695-4433-808d-eec24f33c963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:23.162160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:23.164321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 516/1025] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:23.451596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc788a-3e02-4b86-a36b-5aa5baa60d32 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:23.453250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-93cc788a-3e02-4b86-a36b-5aa5baa60d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:23.555139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc788a-3e02-4b86-a36b-5aa5baa60d32 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:23.606267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:23.632141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:23.632757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:23.633059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:23.633333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:23.635001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:23.745215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:23.746588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db67460-088d-46b5-ac3a-1f1aa35e2837 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:23.765407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 510/1026] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:28.665941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d415-a00e-43fa-a8ad-3d09aedd0091 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:28.666408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d415-a00e-43fa-a8ad-3d09aedd0091 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:28.666495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d415-a00e-43fa-a8ad-3d09aedd0091 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:28.668043 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 517/1027] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:28.673613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da49cc-1bc5-441b-a858-a1eeb781567e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:28.674055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da49cc-1bc5-441b-a858-a1eeb781567e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:28.674142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da49cc-1bc5-441b-a858-a1eeb781567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:28.827289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f4da49cc-1bc5-441b-a858-a1eeb781567e None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:28.828549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f4da49cc-1bc5-441b-a858-a1eeb781567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:28.888558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:29.086358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 511/1028] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:00:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:33.093797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0d1d7-353a-462f-87f6-edbc018b309a None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:33.094310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0d1d7-353a-462f-87f6-edbc018b309a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:33.094310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0d1d7-353a-462f-87f6-edbc018b309a None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:33.096251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 518/1029] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:33.105175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84e55-bef2-4ca6-a38f-89202e005819 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:33.105613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84e55-bef2-4ca6-a38f-89202e005819 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:33.105706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f84e55-bef2-4ca6-a38f-89202e005819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:33.183382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d3f84e55-bef2-4ca6-a38f-89202e005819 None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:33.184608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d3f84e55-bef2-4ca6-a38f-89202e005819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:33.260439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:33.295664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 512/1030] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:00:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:34.670085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128373c-5843-431d-a816-0c79c890a69c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:34.670620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128373c-5843-431d-a816-0c79c890a69c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:34.670750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128373c-5843-431d-a816-0c79c890a69c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:34.672229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 519/1031] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:34.677258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89950619-c87a-4bf3-8a72-4aeb3cf9ad03 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:34.677684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89950619-c87a-4bf3-8a72-4aeb3cf9ad03 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:34.677810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89950619-c87a-4bf3-8a72-4aeb3cf9ad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:34.759593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-89950619-c87a-4bf3-8a72-4aeb3cf9ad03 None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:34.760827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-89950619-c87a-4bf3-8a72-4aeb3cf9ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:34.873763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 513/1032] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:00:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:38.522313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e47f-126a-4fa7-9fc4-b60c03573443 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:38.522841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e47f-126a-4fa7-9fc4-b60c03573443 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:38.522841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e47f-126a-4fa7-9fc4-b60c03573443 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:38.528148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 520/1033] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:38.532214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0d688-3887-477d-ada0-3f4f813d61f1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:38.532664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0d688-3887-477d-ada0-3f4f813d61f1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:38.532765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0d688-3887-477d-ada0-3f4f813d61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:38.570463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:38.603252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-98a0d688-3887-477d-ada0-3f4f813d61f1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:38.604507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-98a0d688-3887-477d-ada0-3f4f813d61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:38.706290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 514/1034] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:39.007393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687486-d3c9-436f-b804-93f7b7f0bb43 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:39.007750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687486-d3c9-436f-b804-93f7b7f0bb43 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:39.008204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687486-d3c9-436f-b804-93f7b7f0bb43 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:39.009616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 521/1035] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 10 17:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:39.017558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed1ee-4604-4147-92ef-a154e4be4633 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:39.018040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed1ee-4604-4147-92ef-a154e4be4633 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:39.019004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed1ee-4604-4147-92ef-a154e4be4633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:39.104253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-360ed1ee-4604-4147-92ef-a154e4be4633 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:39.105542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-360ed1ee-4604-4147-92ef-a154e4be4633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:39.158290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:39.227356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 515/1036] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 10 17:00:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:00:39.235267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687486-d3c9-436f-b804-93f7b7f0bb43 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:39.236788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d8687486-d3c9-436f-b804-93f7b7f0bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:39.389977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687486-d3c9-436f-b804-93f7b7f0bb43 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:39.467920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:39.509924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:39.510520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:39.510799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:39.511064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:39.512658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:39.613295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:39.614881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2059e0-a531-4a0a-a716-778fe432d85a service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:39.635565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 522/1037] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:42.060648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ec04-3231-4a34-a6a9-99362e8f128f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:42.061303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ec04-3231-4a34-a6a9-99362e8f128f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:42.061303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ec04-3231-4a34-a6a9-99362e8f128f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:42.067097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 516/1038] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:42.068721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0be74-f04b-46b5-8ea1-e3af1f9bea47 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:42.068721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0be74-f04b-46b5-8ea1-e3af1f9bea47 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:42.069292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0be74-f04b-46b5-8ea1-e3af1f9bea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:42.143766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-08e0be74-f04b-46b5-8ea1-e3af1f9bea47 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:42.145095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-08e0be74-f04b-46b5-8ea1-e3af1f9bea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:42.190501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:42.251142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 523/1039] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:00:42.549067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487ec04-3231-4a34-a6a9-99362e8f128f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:42.550626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6487ec04-3231-4a34-a6a9-99362e8f128f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:42.728460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487ec04-3231-4a34-a6a9-99362e8f128f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:42.744111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:42.855466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:42.856091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:42.856375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:42.856640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:42.858225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:42.959653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:42.961203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac499b-821d-40dd-b962-6028c3724373 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:42.979702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 517/1040] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:45.586596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965bb6-ed85-4a26-b210-bbf502f621f4 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:45.586998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965bb6-ed85-4a26-b210-bbf502f621f4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:45.587136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965bb6-ed85-4a26-b210-bbf502f621f4 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:45.589031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 524/1041] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:45.594694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba27e5-f397-48e2-8599-24ff7ecca18e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:45.595118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba27e5-f397-48e2-8599-24ff7ecca18e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:45.595287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba27e5-f397-48e2-8599-24ff7ecca18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:45.612440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:45.664579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bdba27e5-f397-48e2-8599-24ff7ecca18e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:45.665785 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bdba27e5-f397-48e2-8599-24ff7ecca18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:45.769302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 518/1042] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:46.060557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e58f1a-367d-4de0-8fe4-f1878f99a23b None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:46.060844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e58f1a-367d-4de0-8fe4-f1878f99a23b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:46.061118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e58f1a-367d-4de0-8fe4-f1878f99a23b None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:46.062477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 525/1043] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 10 17:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:46.068757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ab3cb-9bd9-4119-a80f-6ad381138a3f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:46.069046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ab3cb-9bd9-4119-a80f-6ad381138a3f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:46.069320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ab3cb-9bd9-4119-a80f-6ad381138a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:46.148372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-806ab3cb-9bd9-4119-a80f-6ad381138a3f None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:46.149493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-806ab3cb-9bd9-4119-a80f-6ad381138a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:46.185999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:46.274386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 519/1044] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 10 17:00:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:46.282866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e58f1a-367d-4de0-8fe4-f1878f99a23b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:46.284542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f6e58f1a-367d-4de0-8fe4-f1878f99a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:46.448812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e58f1a-367d-4de0-8fe4-f1878f99a23b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:46.470775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:46.575952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:46.576600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:46.576879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:46.577152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:46.579266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:46.624215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de8e77-8af0-4775-b63d-8562139da692 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:46.624787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de8e77-8af0-4775-b63d-8562139da692 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:46.624787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de8e77-8af0-4775-b63d-8562139da692 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:46.626311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 520/1045] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 17:00:46.631053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f28e-e63b-4399-afa3-2e3e60b24f0a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:46.631338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f28e-e63b-4399-afa3-2e3e60b24f0a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:46.631617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f28e-e63b-4399-afa3-2e3e60b24f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:46.673520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:46.675900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3f452-f56e-4897-a6f0-b92e9e25850d service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:46.695639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 526/1046] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:46.707541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2b83f28e-e63b-4399-afa3-2e3e60b24f0a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:46.708863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2b83f28e-e63b-4399-afa3-2e3e60b24f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:46.810260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:46.818134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 521/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:00:47.171924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50052a4-846b-4d37-8a22-981db1541344 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:47.172305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50052a4-846b-4d37-8a22-981db1541344 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:47.172793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50052a4-846b-4d37-8a22-981db1541344 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:47.174310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 527/1048] 10.222.0.42 () {54 vars in 963 bytes} [Mon Oct 10 17:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:47.179713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9fc05-4160-4ad3-b157-e8b374c604f8 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:47.180203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9fc05-4160-4ad3-b157-e8b374c604f8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:47.180727 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9fc05-4160-4ad3-b157-e8b374c604f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:47.261239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a5d9fc05-4160-4ad3-b157-e8b374c604f8 None None] MFA Rules not processed for user `eff532d7c1bc4c02838873c67fca787f`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:47.262610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a5d9fc05-4160-4ad3-b157-e8b374c604f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:47.345472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 522/1049] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Oct 10 17:00:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:00:47.355596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50052a4-846b-4d37-8a22-981db1541344 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:47.357139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f50052a4-846b-4d37-8a22-981db1541344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:47.387148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:47.468074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50052a4-846b-4d37-8a22-981db1541344 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:47.551645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:47.553360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:47.553635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:47.553907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:47.555549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:47.636865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:47.677431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:47.679355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d689624-b671-4ca9-bfd3-d12c00065600 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:47.702631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 528/1050] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:49.159420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a282b-3210-4670-8cec-a6dfd6ddb0a7 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:49.159910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a282b-3210-4670-8cec-a6dfd6ddb0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:49.160004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a282b-3210-4670-8cec-a6dfd6ddb0a7 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:49.161351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 523/1051] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:49.166563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ce579-8e46-437e-ab77-64a79a95536f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:49.166993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ce579-8e46-437e-ab77-64a79a95536f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:49.167119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ce579-8e46-437e-ab77-64a79a95536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:49.233026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-957ce579-8e46-437e-ab77-64a79a95536f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:49.234242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-957ce579-8e46-437e-ab77-64a79a95536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:49.333846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 529/1052] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:49.626400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a282b-3210-4670-8cec-a6dfd6ddb0a7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:49.627934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-496a282b-3210-4670-8cec-a6dfd6ddb0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:49.715204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:49.781400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a282b-3210-4670-8cec-a6dfd6ddb0a7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:49.901569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:49.902200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:49.902482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:49.902762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:49.904395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:49.959063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:50.002192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:50.003784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a869e0c-1ca8-494f-9090-e9bf7fdb74b8 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:50.024366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 524/1053] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:52.377768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cc59d-c8f3-4920-9a8d-d6e90b97c326 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:52.378337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cc59d-c8f3-4920-9a8d-d6e90b97c326 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:52.378337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cc59d-c8f3-4920-9a8d-d6e90b97c326 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:52.379776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 530/1054] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:52.384955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05526bc-20f6-4e43-9214-07c88a55bd32 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:52.385345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05526bc-20f6-4e43-9214-07c88a55bd32 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:52.385505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05526bc-20f6-4e43-9214-07c88a55bd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:52.455202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f05526bc-20f6-4e43-9214-07c88a55bd32 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:52.456423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f05526bc-20f6-4e43-9214-07c88a55bd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:52.560317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 525/1055] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:52.850206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cc59d-c8f3-4920-9a8d-d6e90b97c326 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:52.851746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4f4cc59d-c8f3-4920-9a8d-d6e90b97c326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:52.890670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:53.008425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cc59d-c8f3-4920-9a8d-d6e90b97c326 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:53.128233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:53.136475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:53.143529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:53.143529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:53.143529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:53.143529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:53.236431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:53.238003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5d873-56ba-4739-82c8-85c8cff56a75 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:53.257498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 531/1056] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:55.580632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0e4e3-053f-4720-b297-98c9a148c4fd None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:55.581237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0e4e3-053f-4720-b297-98c9a148c4fd None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:55.581237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0e4e3-053f-4720-b297-98c9a148c4fd None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:55.582537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 526/1057] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:55.588287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a276440-5d8a-45c0-b983-e4af90d3f3fd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:55.588740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a276440-5d8a-45c0-b983-e4af90d3f3fd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:55.588944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a276440-5d8a-45c0-b983-e4af90d3f3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:55.658107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4a276440-5d8a-45c0-b983-e4af90d3f3fd None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:55.659248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4a276440-5d8a-45c0-b983-e4af90d3f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:55.735197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:55.773585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 532/1058] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:00:56.127745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0e4e3-053f-4720-b297-98c9a148c4fd None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:56.129326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-31e0e4e3-053f-4720-b297-98c9a148c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:56.158629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:56.292564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0e4e3-053f-4720-b297-98c9a148c4fd None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:56.412354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:56.431919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:56.432686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:56.433081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:56.433358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:56.435433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:56.529321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:56.530985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb02f1-7000-48cc-9128-7d75d699c7b4 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:00:56.550661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 527/1059] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:00:59.095624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2bbd1-b533-45de-9fe4-d69e16a5eb61 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:59.096343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2bbd1-b533-45de-9fe4-d69e16a5eb61 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:59.096535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2bbd1-b533-45de-9fe4-d69e16a5eb61 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:59.098347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 533/1060] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:00:59.104098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c78c6c-edd8-4e09-9848-f9f4174c730d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:59.104716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c78c6c-edd8-4e09-9848-f9f4174c730d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:59.104826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c78c6c-edd8-4e09-9848-f9f4174c730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:59.186646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-82c78c6c-edd8-4e09-9848-f9f4174c730d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:00:59.189078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-82c78c6c-edd8-4e09-9848-f9f4174c730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:59.260572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:59.301581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 528/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:00:59.598957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2bbd1-b533-45de-9fe4-d69e16a5eb61 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:00:59.600607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f7c2bbd1-b533-45de-9fe4-d69e16a5eb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:59.750632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2bbd1-b533-45de-9fe4-d69e16a5eb61 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:00:59.808966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:00:59.877294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:00:59.877909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:00:59.878196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:00:59.878468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:00:59.880129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:00:59.985127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:00:59.987098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f61ac-1647-4d43-9d3b-b21bc078a93f service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:00.009603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 534/1062] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:02.218880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c045681-f27e-4ad0-ba33-914cc57275d6 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:02.219433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c045681-f27e-4ad0-ba33-914cc57275d6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:02.219433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c045681-f27e-4ad0-ba33-914cc57275d6 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:02.220957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 529/1063] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:02.226077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a11ec8-47a3-477f-a028-385ffebedc92 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:02.226479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a11ec8-47a3-477f-a028-385ffebedc92 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:02.226762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a11ec8-47a3-477f-a028-385ffebedc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:02.292340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-37a11ec8-47a3-477f-a028-385ffebedc92 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:02.293554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-37a11ec8-47a3-477f-a028-385ffebedc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:02.302367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:02.391284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 535/1064] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:02.683680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c045681-f27e-4ad0-ba33-914cc57275d6 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:02.685240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3c045681-f27e-4ad0-ba33-914cc57275d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:02.830822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c045681-f27e-4ad0-ba33-914cc57275d6 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:02.880003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:02.946528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:02.947044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:02.947317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:02.947584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:02.949191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:03.039524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:03.041129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3e03b-a48d-4656-9eb3-7ebc2963cfd1 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:03.059452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 530/1065] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:03.201076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64f63-8d81-438e-809c-a1f49dd41181 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:03.201311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64f63-8d81-438e-809c-a1f49dd41181 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:03.201586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64f63-8d81-438e-809c-a1f49dd41181 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:03.202942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 536/1066] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:03.209300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b5eb-87c8-493c-83d3-83dcba4668e2 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:03.211495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b5eb-87c8-493c-83d3-83dcba4668e2 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:03.211495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b5eb-87c8-493c-83d3-83dcba4668e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:03.279068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:03.284685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a1f1b5eb-87c8-493c-83d3-83dcba4668e2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:03.285921 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a1f1b5eb-87c8-493c-83d3-83dcba4668e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:03.393010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 531/1067] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:03.696045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a5a23-abe4-4bfe-974f-852bad74e426 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:03.696520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a5a23-abe4-4bfe-974f-852bad74e426 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:03.696705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a5a23-abe4-4bfe-974f-852bad74e426 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:03.698092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 537/1068] 10.222.0.42 () {54 vars in 962 bytes} [Mon Oct 10 17:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:03.703841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaf4db-9733-42ee-aca6-de7df61b559c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:03.704392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaf4db-9733-42ee-aca6-de7df61b559c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:03.704798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaf4db-9733-42ee-aca6-de7df61b559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:03.779250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e4eaf4db-9733-42ee-aca6-de7df61b559c None None] MFA Rules not processed for user `dfe6839ce3ac4e6eb27b54eb1870c67d`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:03.780483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e4eaf4db-9733-42ee-aca6-de7df61b559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:03.849935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:03.855677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 532/1069] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Oct 10 17:01:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:03.869964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a5a23-abe4-4bfe-974f-852bad74e426 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:03.869964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-bd2a5a23-abe4-4bfe-974f-852bad74e426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:03.971910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a5a23-abe4-4bfe-974f-852bad74e426 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:04.010119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:04.045534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:04.046144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:04.046445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:04.046710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:04.049679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:04.159931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:04.161329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c837574-1030-41c8-ab0e-cd1da896c41e service glance] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:04.181110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 538/1070] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:05.049371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35521fe-e913-466b-83dd-ecfd839ae267 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:05.049895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35521fe-e913-466b-83dd-ecfd839ae267 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:05.049999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35521fe-e913-466b-83dd-ecfd839ae267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:05.118487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a35521fe-e913-466b-83dd-ecfd839ae267 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:05.119693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a35521fe-e913-466b-83dd-ecfd839ae267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:05.204778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 533/1071] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:05.257510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909070c9-bfbd-4e67-967e-a2a6823695c3 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:05.258034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909070c9-bfbd-4e67-967e-a2a6823695c3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:05.258034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909070c9-bfbd-4e67-967e-a2a6823695c3 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:05.259423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 539/1072] 10.222.0.42 () {54 vars in 943 bytes} [Mon Oct 10 17:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:05.265851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:05.266138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:05.266414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:05.341032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c18179-9d2c-47c5-89f9-901d23cf96f0 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:05.341305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c18179-9d2c-47c5-89f9-901d23cf96f0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:05.341580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c18179-9d2c-47c5-89f9-901d23cf96f0 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:05.343372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 540/1073] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 17:01:05.344962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] MFA Rules not processed for user `c3e348fd8e2b4203b7bc43240cb79d74`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:05.346229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:05.349156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562d805-4952-4766-9167-6c380b549c59 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:05.349758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562d805-4952-4766-9167-6c380b549c59 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:05.350028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562d805-4952-4766-9167-6c380b549c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:05.373618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:05.416888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8562d805-4952-4766-9167-6c380b549c59 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:05.418127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8562d805-4952-4766-9167-6c380b549c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:05.436297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:05.451786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 534/1074] 10.222.0.42 () {58 vars in 1040 bytes} [Mon Oct 10 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:05.461780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:05.471833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c411c99-fe7a-4fbe-8ec3-3830866801dd None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:05.517403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 541/1075] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:01:05.573441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:05.574024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:05.574302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:05.574586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:05.576226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:05.609000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:05.658176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:05.659756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0391d58-386b-40bb-83b3-866440471ed9 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:05.667781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 535/1076] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:01:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 17:01:05.805785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562d805-4952-4766-9167-6c380b549c59 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:05.807033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8562d805-4952-4766-9167-6c380b549c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:05.920715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562d805-4952-4766-9167-6c380b549c59 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:05.958125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:06.050319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:06.050910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:06.051193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:06.053088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:06.053088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:06.149526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:06.151216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99222a43-d5e1-4ec8-801a-20eb0725fca0 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:06.188900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 542/1077] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:09.053719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b294-a3dd-4ba3-84b2-84edf9a6a772 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:09.054302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b294-a3dd-4ba3-84b2-84edf9a6a772 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:09.054302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b294-a3dd-4ba3-84b2-84edf9a6a772 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:09.055779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 536/1078] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:09.060999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc04f-a4d1-4d97-91a7-86fd530b0130 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:09.064318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc04f-a4d1-4d97-91a7-86fd530b0130 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:09.064318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc04f-a4d1-4d97-91a7-86fd530b0130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:09.148746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:09.167091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ee5bc04f-a4d1-4d97-91a7-86fd530b0130 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:09.170358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ee5bc04f-a4d1-4d97-91a7-86fd530b0130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:09.299727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 543/1079] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:01:09.590948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b294-a3dd-4ba3-84b2-84edf9a6a772 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:09.592485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1881b294-a3dd-4ba3-84b2-84edf9a6a772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:09.756356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b294-a3dd-4ba3-84b2-84edf9a6a772 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:09.787110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:09.895036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:09.902193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:09.902193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:09.902193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:09.902193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:10.003934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:10.005585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4351560-e951-4d8a-a9ea-379cb3b95c03 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:10.030082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 537/1080] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:12.402263 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452c993-0146-419a-bf18-06c7433574f0 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:12.404978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452c993-0146-419a-bf18-06c7433574f0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:12.404978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452c993-0146-419a-bf18-06c7433574f0 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:12.405456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 544/1081] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:12.410272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0edbf-91f4-432f-8ec9-a8d47a7e6f70 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:12.410716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0edbf-91f4-432f-8ec9-a8d47a7e6f70 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:12.417116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0edbf-91f4-432f-8ec9-a8d47a7e6f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:12.459508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:12.492333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-47f0edbf-91f4-432f-8ec9-a8d47a7e6f70 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:12.493658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-47f0edbf-91f4-432f-8ec9-a8d47a7e6f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:12.599803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 538/1082] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:12.894652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452c993-0146-419a-bf18-06c7433574f0 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:12.896260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0452c993-0146-419a-bf18-06c7433574f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:13.021558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:13.059339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452c993-0146-419a-bf18-06c7433574f0 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:13.183203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:13.183929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:13.186002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:13.188664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:13.190487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:13.282143 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:13.293144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:13.302603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a172f-2a52-4065-9cc8-17536f7c6fd0 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:13.316357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 545/1083] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:15.680676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ba9e2-86ae-4d65-a346-39b59139fb9e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:15.681613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ba9e2-86ae-4d65-a346-39b59139fb9e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:15.682095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ba9e2-86ae-4d65-a346-39b59139fb9e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:15.683452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 539/1084] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:15.689062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66249195-e13e-47dd-9f4a-9c318a37f2fb None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:15.689522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66249195-e13e-47dd-9f4a-9c318a37f2fb None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:15.689629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66249195-e13e-47dd-9f4a-9c318a37f2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:15.761337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-66249195-e13e-47dd-9f4a-9c318a37f2fb None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:15.762624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-66249195-e13e-47dd-9f4a-9c318a37f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:15.882567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 546/1085] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:16.166008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ba9e2-86ae-4d65-a346-39b59139fb9e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:16.167525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c80ba9e2-86ae-4d65-a346-39b59139fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:16.283491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:16.327425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ba9e2-86ae-4d65-a346-39b59139fb9e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:16.455058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:16.455778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:16.456160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:16.456499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:16.458126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:16.537298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:16.554228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:16.555871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7a486-8201-4908-b7d1-f271532717f0 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:16.575318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 540/1086] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:25.867296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=74 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:26.573161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=13 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:29.569579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6d5f2-9cd6-4b3b-8c1f-8543ce46af01 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:29.570151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6d5f2-9cd6-4b3b-8c1f-8543ce46af01 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:29.570151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6d5f2-9cd6-4b3b-8c1f-8543ce46af01 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:29.571468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 547/1087] 10.222.0.42 () {54 vars in 918 bytes} [Mon Oct 10 17:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:29.576565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a4d00-e8d6-49e0-bf2b-c0fa54c03b95 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:29.577035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a4d00-e8d6-49e0-bf2b-c0fa54c03b95 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:29.577151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a4d00-e8d6-49e0-bf2b-c0fa54c03b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:29.653799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-793a4d00-e8d6-49e0-bf2b-c0fa54c03b95 None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:29.655005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-793a4d00-e8d6-49e0-bf2b-c0fa54c03b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:29.759888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 541/1088] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Oct 10 17:01:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:29.774010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2499d8-f305-4ca4-afa2-39538552b355 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:29.774292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2499d8-f305-4ca4-afa2-39538552b355 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:29.774564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2499d8-f305-4ca4-afa2-39538552b355 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:29.775880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 548/1089] 10.222.0.42 () {54 vars in 918 bytes} [Mon Oct 10 17:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:29.780493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3a1b-df28-4bb1-84e4-fc587b1b67e5 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:29.781287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3a1b-df28-4bb1-84e4-fc587b1b67e5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:29.781553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3a1b-df28-4bb1-84e4-fc587b1b67e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:29.853291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:29.855573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e4fa3a1b-df28-4bb1-84e4-fc587b1b67e5 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:29.856795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e4fa3a1b-df28-4bb1-84e4-fc587b1b67e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:29.970539 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 542/1090] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Oct 10 17:01:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:32.541821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879d6a0-0402-4750-9fb2-3cf930631ae1 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:32.542389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879d6a0-0402-4750-9fb2-3cf930631ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:32.542389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879d6a0-0402-4750-9fb2-3cf930631ae1 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:32.543686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 549/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:32.549059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123440b5-d37a-4f0e-8cc8-b96add89b370 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:32.549460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123440b5-d37a-4f0e-8cc8-b96add89b370 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:32.549615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123440b5-d37a-4f0e-8cc8-b96add89b370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:32.614414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-123440b5-d37a-4f0e-8cc8-b96add89b370 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:32.615608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-123440b5-d37a-4f0e-8cc8-b96add89b370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:32.669011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:32.710643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 543/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:33.000432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879d6a0-0402-4750-9fb2-3cf930631ae1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:33.002042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c879d6a0-0402-4750-9fb2-3cf930631ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:33.146136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879d6a0-0402-4750-9fb2-3cf930631ae1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:33.216969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:33.261588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:33.262198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:33.262478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:33.262750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:33.264373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:33.352666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:33.354235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b47fa-e66a-48b2-96b3-824f22d3748c service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:33.372356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 550/1093] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:34.502730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93ad4f-29fd-4662-9250-66b4fc31be5f None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:34.503275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93ad4f-29fd-4662-9250-66b4fc31be5f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:34.503275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93ad4f-29fd-4662-9250-66b4fc31be5f None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:34.504653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 544/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:34.510915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606966b4-7ca6-40ca-976b-487c6737a8fa None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:34.511384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606966b4-7ca6-40ca-976b-487c6737a8fa None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:34.511483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606966b4-7ca6-40ca-976b-487c6737a8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:34.586671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-606966b4-7ca6-40ca-976b-487c6737a8fa None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:34.587869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-606966b4-7ca6-40ca-976b-487c6737a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:34.614571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:34.681469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 551/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:34.831914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93ad4f-29fd-4662-9250-66b4fc31be5f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:34.833378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-df93ad4f-29fd-4662-9250-66b4fc31be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:34.981032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93ad4f-29fd-4662-9250-66b4fc31be5f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:35.020841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:35.093199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:35.093717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:35.093993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:35.094258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:35.095861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:35.180420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:35.181982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93f9b6-4d5f-4bba-9be4-a32fbe5c1469 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:35.199858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 545/1096] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:36.323513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7efff7-5a96-494e-8dce-4b73619a8e2b None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:36.324093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7efff7-5a96-494e-8dce-4b73619a8e2b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:36.324093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7efff7-5a96-494e-8dce-4b73619a8e2b None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:36.325481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 552/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:36.331697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909406-eba1-4c1e-9d36-48ad1d52e256 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:36.332129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909406-eba1-4c1e-9d36-48ad1d52e256 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:36.332275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909406-eba1-4c1e-9d36-48ad1d52e256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:36.385276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:36.406350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-41909406-eba1-4c1e-9d36-48ad1d52e256 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:36.410435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-41909406-eba1-4c1e-9d36-48ad1d52e256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:36.502852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 546/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:36.654122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7efff7-5a96-494e-8dce-4b73619a8e2b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:36.656210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0d7efff7-5a96-494e-8dce-4b73619a8e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:36.798172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7efff7-5a96-494e-8dce-4b73619a8e2b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:36.809738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:36.909962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:36.910497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:36.910770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:36.911034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:36.912622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:36.999199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:37.000846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55aef5-36a6-444e-9688-738fe4420fd1 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:37.018613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 553/1099] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:38.117397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9ead9-3849-4144-a60e-b9216221dc05 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:38.117993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9ead9-3849-4144-a60e-b9216221dc05 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:38.117993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9ead9-3849-4144-a60e-b9216221dc05 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:38.119383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 547/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:38.125704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b497ab-0300-477f-aa0d-97e0d456650c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:38.126185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b497ab-0300-477f-aa0d-97e0d456650c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:38.126295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b497ab-0300-477f-aa0d-97e0d456650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:38.145627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:38.204427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a0b497ab-0300-477f-aa0d-97e0d456650c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:38.205617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a0b497ab-0300-477f-aa0d-97e0d456650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:38.301069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 554/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:38.450018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9ead9-3849-4144-a60e-b9216221dc05 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:38.451566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-79e9ead9-3849-4144-a60e-b9216221dc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:38.558938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:38.594729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9ead9-3849-4144-a60e-b9216221dc05 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:38.705505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:38.706043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:38.706322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:38.706591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:38.708211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:38.781358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:38.794639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:38.796264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf7eb5-c13b-4f16-86f6-dc94edabcf80 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:38.814585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 548/1102] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:39.913875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b50f3-658a-429e-93e7-2f895b0626f2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:39.914593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b50f3-658a-429e-93e7-2f895b0626f2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:39.914593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b50f3-658a-429e-93e7-2f895b0626f2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:39.915797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 555/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:39.922369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb46e0-b4c9-4dd8-9279-0f0cac3aedb6 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:39.922825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb46e0-b4c9-4dd8-9279-0f0cac3aedb6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:39.922939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb46e0-b4c9-4dd8-9279-0f0cac3aedb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:40.000131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8dbb46e0-b4c9-4dd8-9279-0f0cac3aedb6 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:40.001393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8dbb46e0-b4c9-4dd8-9279-0f0cac3aedb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:40.096322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 549/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:40.244944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b50f3-658a-429e-93e7-2f895b0626f2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:40.246416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-193b50f3-658a-429e-93e7-2f895b0626f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:40.331521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:40.392256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b50f3-658a-429e-93e7-2f895b0626f2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:40.503148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:40.503677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:40.503959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:40.504261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:40.505857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:40.553490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:40.594122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:40.595824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c88a7-e9d8-49f3-be40-20871d99e082 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:40.614098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 556/1105] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:41.710429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ef2a4-cda2-4fc2-810b-c2dd1e825304 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:41.710982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ef2a4-cda2-4fc2-810b-c2dd1e825304 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:41.710982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ef2a4-cda2-4fc2-810b-c2dd1e825304 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:41.712492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 550/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:41.718781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bf7b5-40d9-4a1c-8985-ca000b5e7147 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:41.719243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bf7b5-40d9-4a1c-8985-ca000b5e7147 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:41.719360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bf7b5-40d9-4a1c-8985-ca000b5e7147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:41.793820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5c9bf7b5-40d9-4a1c-8985-ca000b5e7147 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:41.795035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5c9bf7b5-40d9-4a1c-8985-ca000b5e7147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:41.891034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 557/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:42.040339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ef2a4-cda2-4fc2-810b-c2dd1e825304 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:42.041860 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e58ef2a4-cda2-4fc2-810b-c2dd1e825304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:42.091072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:42.184971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ef2a4-cda2-4fc2-810b-c2dd1e825304 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:42.297476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:42.298086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:42.298366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:42.298640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:42.300283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:42.307096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:42.388133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:42.389718 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841800d-cbcd-4ce3-9ae7-b09a00fd38ac service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:42.407744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 551/1108] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:43.516951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe45e14-564c-4f33-9807-63b0f31db51c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:43.517484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe45e14-564c-4f33-9807-63b0f31db51c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:43.517484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe45e14-564c-4f33-9807-63b0f31db51c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:43.518813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 558/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:43.525073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35bf62-5603-4e13-a9b7-87956b77568f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:43.525512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35bf62-5603-4e13-a9b7-87956b77568f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:43.525628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35bf62-5603-4e13-a9b7-87956b77568f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:43.600656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-df35bf62-5603-4e13-a9b7-87956b77568f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:43.601858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-df35bf62-5603-4e13-a9b7-87956b77568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:43.669074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:43.698601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 552/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:43.845847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe45e14-564c-4f33-9807-63b0f31db51c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:43.847312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cfe45e14-564c-4f33-9807-63b0f31db51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:43.864953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:43.992263 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe45e14-564c-4f33-9807-63b0f31db51c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:44.077270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:44.102994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:44.103534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:44.103815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:44.104117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:44.105710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:44.194699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:44.196296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f503872-0d43-437c-b615-c4b0c1b08d58 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:44.213680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 559/1111] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:45.315743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9737-a599-456d-a87f-a7a59f8ad4d0 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:45.316343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9737-a599-456d-a87f-a7a59f8ad4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:45.316343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9737-a599-456d-a87f-a7a59f8ad4d0 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:45.317765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 553/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:45.324283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33dc10-57da-441e-b251-1aa0cfa545dd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:45.324712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33dc10-57da-441e-b251-1aa0cfa545dd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:45.324825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33dc10-57da-441e-b251-1aa0cfa545dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:45.399235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-fd33dc10-57da-441e-b251-1aa0cfa545dd None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:45.400461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fd33dc10-57da-441e-b251-1aa0cfa545dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:45.442311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:45.496099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 560/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:01:45.645424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa9737-a599-456d-a87f-a7a59f8ad4d0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:45.646958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f3fa9737-a599-456d-a87f-a7a59f8ad4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:45.788596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa9737-a599-456d-a87f-a7a59f8ad4d0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:45.841023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:45.898846 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:45.899408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:45.899683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:45.899946 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:45.901550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:45.988338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:45.989924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70294a6c-a582-4ad2-a3f9-6ffe7db7c239 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:46.007676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 554/1114] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:47.113990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c48f-6f3a-4dd4-b692-c5f0d526706b None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:47.114532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c48f-6f3a-4dd4-b692-c5f0d526706b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:47.114532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c48f-6f3a-4dd4-b692-c5f0d526706b None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:47.115778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 561/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:47.121888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38c06-6d06-449e-9b74-d850cdc4ddfa None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:47.122316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38c06-6d06-449e-9b74-d850cdc4ddfa None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:47.122423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38c06-6d06-449e-9b74-d850cdc4ddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:47.196813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8bd38c06-6d06-449e-9b74-d850cdc4ddfa None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:47.198056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8bd38c06-6d06-449e-9b74-d850cdc4ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:47.200112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:47.294294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 555/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:47.444227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3c48f-6f3a-4dd4-b692-c5f0d526706b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:47.445722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a8a3c48f-6f3a-4dd4-b692-c5f0d526706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:47.587182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3c48f-6f3a-4dd4-b692-c5f0d526706b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:47.613321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:47.696677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:47.697226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:47.697496 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:47.697759 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:47.699326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:47.785862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:47.787436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f5a46-6785-4a61-aef4-64fe674fcba6 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:47.804600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 562/1117] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:49.843288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eecaa-8308-48bd-a373-24b38ff670fa None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:49.843876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eecaa-8308-48bd-a373-24b38ff670fa None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:49.843876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eecaa-8308-48bd-a373-24b38ff670fa None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:49.845215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 556/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:49.850530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cd2f5-3843-4479-b088-b667e824641b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:49.851027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cd2f5-3843-4479-b088-b667e824641b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:49.851126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cd2f5-3843-4479-b088-b667e824641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:49.892250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:49.916683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-af6cd2f5-3843-4479-b088-b667e824641b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:49.923050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-af6cd2f5-3843-4479-b088-b667e824641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:50.013237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 563/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:50.301305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9eecaa-8308-48bd-a373-24b38ff670fa None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:50.302827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bf9eecaa-8308-48bd-a373-24b38ff670fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:50.407201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9eecaa-8308-48bd-a373-24b38ff670fa None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:50.438781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:50.479111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:50.479707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:50.479983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:50.480299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:50.481911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:50.585690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:50.587016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe396b0b-ce05-4782-a23c-9b48cc2c2cf2 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:50.605247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 557/1120] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:52.899186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c5cf-844d-465d-8873-fe6af6ce5405 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:52.899702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c5cf-844d-465d-8873-fe6af6ce5405 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:52.899702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c5cf-844d-465d-8873-fe6af6ce5405 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:52.901266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 564/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:52.907044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64fcfa-2a0a-4844-b563-b4f4306baa59 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:52.907472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64fcfa-2a0a-4844-b563-b4f4306baa59 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:52.907570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64fcfa-2a0a-4844-b563-b4f4306baa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:52.971392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ca64fcfa-2a0a-4844-b563-b4f4306baa59 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:52.972628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ca64fcfa-2a0a-4844-b563-b4f4306baa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:52.984994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:53.067752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 558/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:53.349075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806c5cf-844d-465d-8873-fe6af6ce5405 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:53.350653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7806c5cf-844d-465d-8873-fe6af6ce5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:53.453694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806c5cf-844d-465d-8873-fe6af6ce5405 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:53.466344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:53.527155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:53.527763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:53.528049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:53.528333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:53.529916 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:53.637502 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:53.638973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b5dc-5c51-4608-9329-01eb59d3b55c service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:53.657962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 565/1123] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:56.135861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826df7f-cdd4-4abb-bc0d-8396a89cdd83 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:56.136456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826df7f-cdd4-4abb-bc0d-8396a89cdd83 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:56.136456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826df7f-cdd4-4abb-bc0d-8396a89cdd83 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:56.137859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 559/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:56.143633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508daa-6f33-466a-adf4-c3c49398ef8a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:56.144053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508daa-6f33-466a-adf4-c3c49398ef8a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:56.144217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8508daa-6f33-466a-adf4-c3c49398ef8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:56.191704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:56.209699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e8508daa-6f33-466a-adf4-c3c49398ef8a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:56.214287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e8508daa-6f33-466a-adf4-c3c49398ef8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:56.305682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 566/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:56.594209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826df7f-cdd4-4abb-bc0d-8396a89cdd83 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:56.595842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d826df7f-cdd4-4abb-bc0d-8396a89cdd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:56.698827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826df7f-cdd4-4abb-bc0d-8396a89cdd83 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:56.738110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:56.769156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:56.769749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:56.770023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:56.770294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:56.771912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:56.879467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:56.880913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58ca62-a322-4531-9643-c96e92d2312e service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:56.898458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 560/1126] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:01:59.168772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32602a2-c282-4d79-a62e-2f78c8b6c7d2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:59.169297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32602a2-c282-4d79-a62e-2f78c8b6c7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:59.169297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32602a2-c282-4d79-a62e-2f78c8b6c7d2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:59.170688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 567/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:01:59.176227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c243fa5-1480-4840-a832-6dbd3820efc0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:59.176656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c243fa5-1480-4840-a832-6dbd3820efc0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:59.176765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c243fa5-1480-4840-a832-6dbd3820efc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:59.243584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5c243fa5-1480-4840-a832-6dbd3820efc0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:01:59.244832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5c243fa5-1480-4840-a832-6dbd3820efc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:59.264918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:59.340784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 561/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:01:59.636131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32602a2-c282-4d79-a62e-2f78c8b6c7d2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:01:59.637651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e32602a2-c282-4d79-a62e-2f78c8b6c7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:59.738694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32602a2-c282-4d79-a62e-2f78c8b6c7d2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:01:59.758898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:01:59.810752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:01:59.811873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:01:59.812173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:01:59.812440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:01:59.814312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:01:59.922661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:01:59.924238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8d7d5-8fdc-469b-9500-a5734ac3f5ec service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:01:59.942556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 568/1129] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:02.397093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e32eb-04a9-4e83-91b2-a5943a43f34c None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:02.397686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e32eb-04a9-4e83-91b2-a5943a43f34c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:02.397686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e32eb-04a9-4e83-91b2-a5943a43f34c None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:02.399002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 562/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:02.404510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa31c1-8b52-4bff-a1c3-92146de3679e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:02.404928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa31c1-8b52-4bff-a1c3-92146de3679e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:02.405095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa31c1-8b52-4bff-a1c3-92146de3679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:02.468062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:02.470542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-80aa31c1-8b52-4bff-a1c3-92146de3679e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:02.471746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-80aa31c1-8b52-4bff-a1c3-92146de3679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:02.566438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 569/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:02.847327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e32eb-04a9-4e83-91b2-a5943a43f34c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:02.848889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-940e32eb-04a9-4e83-91b2-a5943a43f34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:02.950727 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e32eb-04a9-4e83-91b2-a5943a43f34c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:03.000128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:03.024523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:03.025136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:03.025414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:03.025684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:03.027285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:03.134784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:03.136143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a5f4f-7622-4fb0-a672-3ce7ef285578 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:03.154604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 563/1132] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:05.412256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b646569-5206-455d-b61b-541e79cac139 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:05.412808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b646569-5206-455d-b61b-541e79cac139 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:05.412808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b646569-5206-455d-b61b-541e79cac139 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:05.414180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 570/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:05.420169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dcdde-820b-4923-9668-24e94ea3f739 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:05.421402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dcdde-820b-4923-9668-24e94ea3f739 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:05.421402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dcdde-820b-4923-9668-24e94ea3f739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:05.485948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e52dcdde-820b-4923-9668-24e94ea3f739 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:05.487156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e52dcdde-820b-4923-9668-24e94ea3f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:05.515731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:05.585268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 564/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:05.872884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b646569-5206-455d-b61b-541e79cac139 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:05.874440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3b646569-5206-455d-b61b-541e79cac139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:05.975495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b646569-5206-455d-b61b-541e79cac139 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:06.003444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:06.047742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:06.048374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:06.048658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:06.048933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:06.050515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:06.157679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:06.159043 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c001b-b273-46d3-a178-076f82517631 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:06.176706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 571/1135] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:08.389871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc384-6e3b-45ec-baf0-3e83df90dbe8 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:08.390964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc384-6e3b-45ec-baf0-3e83df90dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:08.390964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc384-6e3b-45ec-baf0-3e83df90dbe8 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:08.392392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 565/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:08.397589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a66fbf-df4d-478a-b267-07a110d03a75 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:08.397997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a66fbf-df4d-478a-b267-07a110d03a75 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:08.398146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a66fbf-df4d-478a-b267-07a110d03a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:08.465187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d9a66fbf-df4d-478a-b267-07a110d03a75 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:08.466395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d9a66fbf-df4d-478a-b267-07a110d03a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:08.475178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:08.565168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 572/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:08.847155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc384-6e3b-45ec-baf0-3e83df90dbe8 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:08.848699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8b0cc384-6e3b-45ec-baf0-3e83df90dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:08.954726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc384-6e3b-45ec-baf0-3e83df90dbe8 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:09.009089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:09.026990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:09.027594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:09.027870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:09.028175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:09.029843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:09.137115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:09.138456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8abf8-524b-46a7-9875-c30a925d32db service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:09.155955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 566/1138] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:11.441326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2526a-abf6-47da-bd6c-56b4db96918c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:11.441889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2526a-abf6-47da-bd6c-56b4db96918c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:11.441889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2526a-abf6-47da-bd6c-56b4db96918c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:11.443217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 573/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:11.448418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dfd4b-e49b-41b5-a0a0-cd8c2487adb2 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:11.450059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dfd4b-e49b-41b5-a0a0-cd8c2487adb2 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:11.450059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dfd4b-e49b-41b5-a0a0-cd8c2487adb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:11.514944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-883dfd4b-e49b-41b5-a0a0-cd8c2487adb2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:11.516716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-883dfd4b-e49b-41b5-a0a0-cd8c2487adb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:11.553975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:11.611237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 567/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:11.893794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2526a-abf6-47da-bd6c-56b4db96918c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:11.895377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6cc2526a-abf6-47da-bd6c-56b4db96918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:11.997410 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2526a-abf6-47da-bd6c-56b4db96918c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:12.035599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:12.071425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:12.072045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:12.072325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:12.072588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:12.074153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:12.178105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:12.179535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ca712-bfd6-4b09-9de7-66c42d5e046f service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:12.198534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 574/1141] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:14.347853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85395ddf-f1b2-471d-889f-2fc35dbfbec0 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:14.348461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85395ddf-f1b2-471d-889f-2fc35dbfbec0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:14.348461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85395ddf-f1b2-471d-889f-2fc35dbfbec0 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:14.349814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 568/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:14.355032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011342a-8d41-49a3-b443-29f31fa71884 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:14.355474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011342a-8d41-49a3-b443-29f31fa71884 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:14.355631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011342a-8d41-49a3-b443-29f31fa71884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:14.422129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6011342a-8d41-49a3-b443-29f31fa71884 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:14.423344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6011342a-8d41-49a3-b443-29f31fa71884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:14.441363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:14.519419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 575/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:14.804995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85395ddf-f1b2-471d-889f-2fc35dbfbec0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:14.806545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-85395ddf-f1b2-471d-889f-2fc35dbfbec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:14.909948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85395ddf-f1b2-471d-889f-2fc35dbfbec0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:14.974704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:14.981014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:14.981609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:14.981884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:14.982151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:14.983729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:15.091191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:15.092565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010ad9-1286-48ac-ad66-16a083ceb9c3 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:15.110693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 569/1144] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:17.258394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1f367-35e7-4a83-a775-69a1be4b641d None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:17.258969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1f367-35e7-4a83-a775-69a1be4b641d None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:17.258969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1f367-35e7-4a83-a775-69a1be4b641d None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:17.260328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 576/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:17.266084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244bd1-bd85-46fa-9f98-84eb4951256b None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:17.266484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244bd1-bd85-46fa-9f98-84eb4951256b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:17.266636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244bd1-bd85-46fa-9f98-84eb4951256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:17.331623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-9b244bd1-bd85-46fa-9f98-84eb4951256b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:17.332870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9b244bd1-bd85-46fa-9f98-84eb4951256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:17.379863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:17.428246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 570/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:17.711119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1f367-35e7-4a83-a775-69a1be4b641d None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:17.712674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d9c1f367-35e7-4a83-a775-69a1be4b641d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:17.813218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1f367-35e7-4a83-a775-69a1be4b641d None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:17.861042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:17.886965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:17.887571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:17.887854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:17.888180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:17.889835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:17.995502 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:17.996947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b5502-3031-4f76-b226-53172117a9d9 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:18.015709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 577/1147] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:27.426692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=22 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:28.012941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=66 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:37.858603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5524c8-c686-4767-af0e-6b30d4dfff2c None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:37.859161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5524c8-c686-4767-af0e-6b30d4dfff2c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:37.859161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5524c8-c686-4767-af0e-6b30d4dfff2c None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:37.860564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 571/1148] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:37.865976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37e0-dc96-44b3-bf4b-36305d466160 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:37.866388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37e0-dc96-44b3-bf4b-36305d466160 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:37.866520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37e0-dc96-44b3-bf4b-36305d466160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:37.932688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-587d37e0-dc96-44b3-bf4b-36305d466160 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:37.933891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-587d37e0-dc96-44b3-bf4b-36305d466160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:38.029223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 578/1149] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:38.324187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b7d22-e88e-4a25-ad0a-26b00572526b None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:38.324497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b7d22-e88e-4a25-ad0a-26b00572526b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:38.324771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b7d22-e88e-4a25-ad0a-26b00572526b None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:38.326159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 572/1150] 10.222.0.42 () {54 vars in 962 bytes} [Mon Oct 10 17:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:38.331280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a12b6-4757-4169-9960-946bb4dd877b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:38.331563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a12b6-4757-4169-9960-946bb4dd877b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:38.331836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a12b6-4757-4169-9960-946bb4dd877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:38.407136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c43a12b6-4757-4169-9960-946bb4dd877b None None] MFA Rules not processed for user `dfe6839ce3ac4e6eb27b54eb1870c67d`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:38.408374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c43a12b6-4757-4169-9960-946bb4dd877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:38.428220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:38.484722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 579/1151] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Oct 10 17:02:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:02:38.492399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b7d22-e88e-4a25-ad0a-26b00572526b None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:38.493900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-836b7d22-e88e-4a25-ad0a-26b00572526b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:38.595228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b7d22-e88e-4a25-ad0a-26b00572526b None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:38.668203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:38.668798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:38.669073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:38.669341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:38.670930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:38.724606 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:38.778330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:38.779749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd4a6d-63b0-4e9e-96c8-d1e2f563b6ac service glance] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:38.797812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 573/1152] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:41.026176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766be93-778b-4b72-bc11-4b7284efdee2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:41.026767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766be93-778b-4b72-bc11-4b7284efdee2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:41.026767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766be93-778b-4b72-bc11-4b7284efdee2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:41.028132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 580/1153] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:41.033821 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea205b58-02ac-4758-abf2-2fe218fb9008 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:41.034242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea205b58-02ac-4758-abf2-2fe218fb9008 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:41.034344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea205b58-02ac-4758-abf2-2fe218fb9008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:41.099233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ea205b58-02ac-4758-abf2-2fe218fb9008 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:41.100467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ea205b58-02ac-4758-abf2-2fe218fb9008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:41.196043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 574/1154] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:41.471864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766be93-778b-4b72-bc11-4b7284efdee2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:41.473405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4766be93-778b-4b72-bc11-4b7284efdee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:41.595449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766be93-778b-4b72-bc11-4b7284efdee2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:41.647814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:41.692049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:41.692646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:41.692924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:41.693193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:41.693827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:41.694966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:02:41.694966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 17:02:41.694966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 17:02:41.696107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dbefd-b795-49ae-9a78-e7f7b2a062e0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:41.715089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 581/1155] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 10 17:02:41 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:02:43.891914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5126b76-ab15-4eea-99f4-4750fb7ca508 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:43.892421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5126b76-ab15-4eea-99f4-4750fb7ca508 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:43.892519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5126b76-ab15-4eea-99f4-4750fb7ca508 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:43.893849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 575/1156] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:43.899074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de32d5d-3f06-4cf5-8106-f61fc99f34b3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:43.899569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de32d5d-3f06-4cf5-8106-f61fc99f34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:43.899682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de32d5d-3f06-4cf5-8106-f61fc99f34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:43.965300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2de32d5d-3f06-4cf5-8106-f61fc99f34b3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:43.966491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2de32d5d-3f06-4cf5-8106-f61fc99f34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:44.060616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 582/1157] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:02:44.356389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5126b76-ab15-4eea-99f4-4750fb7ca508 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:44.357903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d5126b76-ab15-4eea-99f4-4750fb7ca508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:44.367894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:44.503817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5126b76-ab15-4eea-99f4-4750fb7ca508 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:44.585341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:44.617642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:44.618475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:44.618762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:44.619041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:44.620707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:44.708493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:44.710085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba974d56-74aa-4cee-a3fd-35f429183ede service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:44.727557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 576/1158] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:47.057298 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522ac7-103c-4100-ab85-ac3581fe090b None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:47.057860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522ac7-103c-4100-ab85-ac3581fe090b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:47.057860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522ac7-103c-4100-ab85-ac3581fe090b None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:47.059199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 583/1159] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:47.064864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ae3cb-42e2-44ba-ae4f-b175df09fd39 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:47.065249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ae3cb-42e2-44ba-ae4f-b175df09fd39 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:47.065409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ae3cb-42e2-44ba-ae4f-b175df09fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:47.129740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-442ae3cb-42e2-44ba-ae4f-b175df09fd39 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:47.130873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-442ae3cb-42e2-44ba-ae4f-b175df09fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:47.176581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:47.226389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 577/1160] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:47.521442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94522ac7-103c-4100-ab85-ac3581fe090b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:47.523003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-94522ac7-103c-4100-ab85-ac3581fe090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:47.546814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:47.624134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94522ac7-103c-4100-ab85-ac3581fe090b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:47.697200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:47.697804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:47.698079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:47.698346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:47.699952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:47.771734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:47.807977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:47.809606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0977c91-4f1d-4383-b8f7-67b37e09496a service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:47.828612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 584/1161] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:49.990918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de38db3-b0fa-4e30-837a-c8eacdf38e9b None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:49.991432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de38db3-b0fa-4e30-837a-c8eacdf38e9b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:49.991537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de38db3-b0fa-4e30-837a-c8eacdf38e9b None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:49.992864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 578/1162] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:49.998388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8d7b8-8f56-45d0-9d6f-944c2137a7f2 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:50.003610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8d7b8-8f56-45d0-9d6f-944c2137a7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:50.003610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8d7b8-8f56-45d0-9d6f-944c2137a7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:50.066484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ead8d7b8-8f56-45d0-9d6f-944c2137a7f2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:50.067727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ead8d7b8-8f56-45d0-9d6f-944c2137a7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:50.162611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 585/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:50.450961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de38db3-b0fa-4e30-837a-c8eacdf38e9b None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:50.452588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7de38db3-b0fa-4e30-837a-c8eacdf38e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:50.554944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de38db3-b0fa-4e30-837a-c8eacdf38e9b None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:50.626029 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:50.627503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:50.628132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:50.628411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:50.628683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:50.630271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:50.737722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:50.739061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db749c0-aed6-4ce7-8b5f-c4bc49583777 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:50.757126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 579/1164] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:53.009795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da56b-7340-46fc-9ca0-65b7c49cb0cf None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:53.010329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da56b-7340-46fc-9ca0-65b7c49cb0cf None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:53.010329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da56b-7340-46fc-9ca0-65b7c49cb0cf None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:53.011688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 586/1165] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:53.017027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef5aeb-96ec-4988-b18f-c26cfc84e27b None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:53.017468 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef5aeb-96ec-4988-b18f-c26cfc84e27b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:53.017578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef5aeb-96ec-4988-b18f-c26cfc84e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:53.081636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-38ef5aeb-96ec-4988-b18f-c26cfc84e27b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:53.082819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-38ef5aeb-96ec-4988-b18f-c26cfc84e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:53.133732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:53.176755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 580/1166] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:53.464618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da56b-7340-46fc-9ca0-65b7c49cb0cf None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:53.466152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-823da56b-7340-46fc-9ca0-65b7c49cb0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:53.502227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:53.568984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da56b-7340-46fc-9ca0-65b7c49cb0cf None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:53.641221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:53.641829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:53.642110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:53.642383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:53.644007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:53.727945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:53.756813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:53.758233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43dda8-1c72-4e47-882f-e1fc797751a7 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:53.776206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 587/1167] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:02:56.233599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f80e-9d0e-4178-a805-de1660cb97e0 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:56.234183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f80e-9d0e-4178-a805-de1660cb97e0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:56.234183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f80e-9d0e-4178-a805-de1660cb97e0 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:56.235523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 581/1168] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 10 17:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:02:56.240846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce5f09-cf53-42e1-8c6d-12e4337c1c29 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:56.241242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce5f09-cf53-42e1-8c6d-12e4337c1c29 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:56.241402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce5f09-cf53-42e1-8c6d-12e4337c1c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:56.307429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f9ce5f09-cf53-42e1-8c6d-12e4337c1c29 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:02:56.308645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f9ce5f09-cf53-42e1-8c6d-12e4337c1c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:56.402308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 588/1169] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 10 17:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:02:56.691990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f80e-9d0e-4178-a805-de1660cb97e0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:02:56.693548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-05d9f80e-9d0e-4178-a805-de1660cb97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:56.792511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f80e-9d0e-4178-a805-de1660cb97e0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:02:56.863409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:02:56.864096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:02:56.864296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:02:56.864571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:02:56.866142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:02:56.877105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:02:56.976178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:02:56.977537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d582c-6c89-497d-bd8e-661b7b020ccf service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:02:56.996260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 582/1170] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:03:06.400828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=82 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:03:06.993976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=50 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:04:42.304376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.305031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.305031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.389167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:04:42.390361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:42.490983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 589/1171] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:04:42.499476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.499963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.500088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.520012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:04:42.521220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:42.529722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 583/1172] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:04:42.534082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:42.543378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db00e-51d4-4fc1-9281-95ce95c02c3e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:42.547917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:42.548555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.548834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.549105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.549747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:04:42.550047 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:42.551059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:04:42.551059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 17:04:42.551059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 17:04:42.551940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:42.568098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 590/1173] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:04:42.572118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:42.581940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8af99-7677-4c4b-8d94-a9e678260c26 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:42.586572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:42.587170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.587452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.587726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.588415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:42.589514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:04:42.589514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 17:04:42.589514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 17:04:42.590770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:42.743672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 584/1174] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:04:42 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:04:42.748682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:42.758405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3db0a7-2deb-4578-acb8-6bf6425e48e8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:42.806985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:04:42.814211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 17:04:42.817646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 17:04:42.818557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:42.905459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 591/1175] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:04:42 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:04:42.909690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:42.920786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b49640-1879-409e-adb0-8846950835c6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:42.925480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:42.926076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.926353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.926623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.927259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:42.928878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:42.942478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 585/1176] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:04:42.946327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:42.956142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57ce27-7ea6-4c41-be74-cce20ce90f76 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:42.960574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d4127-b95f-436a-a612-635984da5e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:42.961157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d4127-b95f-436a-a612-635984da5e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.961433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d4127-b95f-436a-a612-635984da5e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.961708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d4127-b95f-436a-a612-635984da5e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:42.962338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d4127-b95f-436a-a612-635984da5e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:42.963901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d4127-b95f-436a-a612-635984da5e32 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:42.976828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 592/1177] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:04:42.980730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:42.991030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57a34e-9c72-475d-ab8e-511aa79e43b9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:42.995571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a079b-ebba-450d-9cfa-feea0e2826bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:42.996246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a079b-ebba-450d-9cfa-feea0e2826bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:42.996526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a079b-ebba-450d-9cfa-feea0e2826bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:42.996796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a079b-ebba-450d-9cfa-feea0e2826bb admin admin] PATH_INFO: `/v3/projects/2b4451e147d8432c8e99379dd9cb17eb/users/7b76cc399c09491c9a744e7da0f99856/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.011717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a079b-ebba-450d-9cfa-feea0e2826bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4451e147d8432c8e99379dd9cb17eb, user_id=7b76cc399c09491c9a744e7da0f99856, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:43.014455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a079b-ebba-450d-9cfa-feea0e2826bb admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:43.079573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 586/1178] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:04:42 2022] PUT /identity/v3/projects/2b4451e147d8432c8e99379dd9cb17eb/users/7b76cc399c09491c9a744e7da0f99856/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:04:43.092000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330d46d-f428-438b-8aee-7a9b5e01b7b6 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.092332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330d46d-f428-438b-8aee-7a9b5e01b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.092603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330d46d-f428-438b-8aee-7a9b5e01b7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.120367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8330d46d-f428-438b-8aee-7a9b5e01b7b6 None None] MFA Rules not processed for user `7b76cc399c09491c9a744e7da0f99856`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:04:43.121571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8330d46d-f428-438b-8aee-7a9b5e01b7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:43.183312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 593/1179] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:04:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:04:43.196587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697ee1-ae93-4797-ad08-681279c1d679 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.196869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697ee1-ae93-4797-ad08-681279c1d679 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.197136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697ee1-ae93-4797-ad08-681279c1d679 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.198456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 587/1180] 10.222.0.42 () {54 vars in 963 bytes} [Mon Oct 10 17:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:04:43.202964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22850ad3-9152-4833-aca3-ae1dfb7c0512 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.203243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22850ad3-9152-4833-aca3-ae1dfb7c0512 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.203509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22850ad3-9152-4833-aca3-ae1dfb7c0512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.262413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-22850ad3-9152-4833-aca3-ae1dfb7c0512 None None] MFA Rules not processed for user `eff532d7c1bc4c02838873c67fca787f`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:04:43.263668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-22850ad3-9152-4833-aca3-ae1dfb7c0512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:43.337165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 594/1181] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Oct 10 17:04:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:04:43.345308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697ee1-ae93-4797-ad08-681279c1d679 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:43.355493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697ee1-ae93-4797-ad08-681279c1d679 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:43.359589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:43.360221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.360499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.360766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.368790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:43.370192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:43.378685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 588/1182] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:04:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:04:43.423581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dd7a3-6369-4060-a774-d845a0280814 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.423863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dd7a3-6369-4060-a774-d845a0280814 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.424766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dd7a3-6369-4060-a774-d845a0280814 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.427139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 595/1183] 10.222.0.42 () {52 vars in 830 bytes} [Mon Oct 10 17:04:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:04:43.430694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:43.491228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e629-6db6-4300-a82f-b6c8c02cc8f8 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:43.495994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:43.496641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.496910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.497187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] PATH_INFO: `/v3/users/7b76cc399c09491c9a744e7da0f99856` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.498297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b76cc399c09491c9a744e7da0f99856)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:43.499297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:04:43.499297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 17:04:43.499297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 17:04:43.500661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:43.723629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b76cc399c09491c9a744e7da0f99856'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:43.724392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b76cc399c09491c9a744e7da0f99856'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:43.793890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b76cc399c09491c9a744e7da0f99856'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:43.817078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b76cc399c09491c9a744e7da0f99856'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:43.817823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b76cc399c09491c9a744e7da0f99856'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:43.828270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:04:43.838146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 589/1184] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:04:43 2022] DELETE /identity/v3/users/7b76cc399c09491c9a744e7da0f99856 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:04:43.843334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:43.843748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:43.843864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:43.862359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:04:43.863541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:43.873192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:04:43.921286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 596/1185] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:04:43.930890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:43.932370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:44.010288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc6e31-c6c2-4343-9789-297f2e60df74 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:44.014480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097e764-1336-450e-9106-70c87d0f5b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:44.015092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e764-1336-450e-9106-70c87d0f5b75 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:44.015372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e764-1336-450e-9106-70c87d0f5b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:44.015646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e764-1336-450e-9106-70c87d0f5b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:44.024621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097e764-1336-450e-9106-70c87d0f5b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:44.026044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097e764-1336-450e-9106-70c87d0f5b75 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:44.035500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 590/1186] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:04:44.473483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:04:44.477314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:04:44.486614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad56fc5-2841-4ca9-9320-7e99a832d53f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:04:44.491216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:04:44.491863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:04:44.492188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:04:44.492390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] PATH_INFO: `/v3/projects/2b4451e147d8432c8e99379dd9cb17eb` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:04:44.493493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4451e147d8432c8e99379dd9cb17eb)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:04:44.498216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:04:44.498216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 17:04:44.498216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 17:04:44.498216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:04:44.610922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4451e147d8432c8e99379dd9cb17eb', 'request_id': 'req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:44.616999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4451e147d8432c8e99379dd9cb17eb', 'request_id': 'req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:44.630939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4451e147d8432c8e99379dd9cb17eb', 'request_id': 'req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:44.709763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] The token cache is being invalidate because project 2b4451e147d8432c8e99379dd9cb17eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:04:44.710051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:44.710750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f6870b3-2dd7-4078-9949-3c0d71c59d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:04:44.712273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 597/1187] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:04:44 2022] DELETE /identity/v3/projects/2b4451e147d8432c8e99379dd9cb17eb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:04:54.031313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=38 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:04:54.706460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:06:08.957493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0464f00-7cad-4b7d-813f-a643e7a3deb8 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:08.958094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0464f00-7cad-4b7d-813f-a643e7a3deb8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:08.958094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0464f00-7cad-4b7d-813f-a643e7a3deb8 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:08.959402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 591/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:08.964445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36dc06-93fe-41c2-a572-05e9ef0bbabd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:08.970131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36dc06-93fe-41c2-a572-05e9ef0bbabd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:08.970131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36dc06-93fe-41c2-a572-05e9ef0bbabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:08.984746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-eb36dc06-93fe-41c2-a572-05e9ef0bbabd None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:08.985927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-eb36dc06-93fe-41c2-a572-05e9ef0bbabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:09.042159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 598/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:09.196318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0464f00-7cad-4b7d-813f-a643e7a3deb8 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:09.198047 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e0464f00-7cad-4b7d-813f-a643e7a3deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:09.301256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0464f00-7cad-4b7d-813f-a643e7a3deb8 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:09.306701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c400b-0870-46db-a9d7-811de2ebcd75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:09.307346 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c400b-0870-46db-a9d7-811de2ebcd75 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:09.307624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c400b-0870-46db-a9d7-811de2ebcd75 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:09.307898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c400b-0870-46db-a9d7-811de2ebcd75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:09.316680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c400b-0870-46db-a9d7-811de2ebcd75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:09.318261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c400b-0870-46db-a9d7-811de2ebcd75 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:09.327463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 592/1190] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:10.399931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f003-ac19-4157-bd91-84054fbc9de8 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:10.400494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f003-ac19-4157-bd91-84054fbc9de8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:10.400494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f003-ac19-4157-bd91-84054fbc9de8 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:10.401818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 599/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:10.407992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b5f73-4edb-4e75-98a4-c97bb1962d27 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:10.408447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b5f73-4edb-4e75-98a4-c97bb1962d27 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:10.408574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b5f73-4edb-4e75-98a4-c97bb1962d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:10.427922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8b7b5f73-4edb-4e75-98a4-c97bb1962d27 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:10.429129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8b7b5f73-4edb-4e75-98a4-c97bb1962d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:10.437358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 593/1192] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:10.589550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552f003-ac19-4157-bd91-84054fbc9de8 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:10.598598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9d22-a696-4ce1-82b1-c0a4c8590c67 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:10.598889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9d22-a696-4ce1-82b1-c0a4c8590c67 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:10.599162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9d22-a696-4ce1-82b1-c0a4c8590c67 None None] PATH_INFO: `/v3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:10.600484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 594/1193] 10.222.0.25 () {54 vars in 925 bytes} [Mon Oct 10 17:06:10 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:06:10.604563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552f003-ac19-4157-bd91-84054fbc9de8 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:10.613704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10b6b-7fa4-4093-99e8-58ab84a8f97a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:10.614308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10b6b-7fa4-4093-99e8-58ab84a8f97a service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:10.614590 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10b6b-7fa4-4093-99e8-58ab84a8f97a service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:10.614864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10b6b-7fa4-4093-99e8-58ab84a8f97a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:10.623603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10b6b-7fa4-4093-99e8-58ab84a8f97a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:10.625297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10b6b-7fa4-4093-99e8-58ab84a8f97a service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:10.630237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89604bc5-431b-4d7a-a006-c7917ad587b5 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:10.630524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89604bc5-431b-4d7a-a006-c7917ad587b5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:10.630796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89604bc5-431b-4d7a-a006-c7917ad587b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:10.634702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 600/1194] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:10.687940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-89604bc5-431b-4d7a-a006-c7917ad587b5 None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:10.689201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-89604bc5-431b-4d7a-a006-c7917ad587b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:10.764887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 595/1195] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Oct 10 17:06:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:06:19.689996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df11e9-db26-43fe-a5bb-ed70005050a1 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:19.690690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df11e9-db26-43fe-a5bb-ed70005050a1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:19.690876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df11e9-db26-43fe-a5bb-ed70005050a1 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:19.692254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 601/1196] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 10 17:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:19.697419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d16e8-d2c0-4925-865b-34c8134ab1bf None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:19.697825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d16e8-d2c0-4925-865b-34c8134ab1bf None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:19.697965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d16e8-d2c0-4925-865b-34c8134ab1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:19.718471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-193d16e8-d2c0-4925-865b-34c8134ab1bf None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:19.719659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-193d16e8-d2c0-4925-865b-34c8134ab1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:19.728192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 596/1197] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 10 17:06:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:19.735754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df11e9-db26-43fe-a5bb-ed70005050a1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:19.748897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df11e9-db26-43fe-a5bb-ed70005050a1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:19.753797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d8b60-c4d6-4e5a-95ca-fe5a96d18b7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:19.754376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d8b60-c4d6-4e5a-95ca-fe5a96d18b7a service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:19.754677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d8b60-c4d6-4e5a-95ca-fe5a96d18b7a service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:19.754945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d8b60-c4d6-4e5a-95ca-fe5a96d18b7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:19.763311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d8b60-c4d6-4e5a-95ca-fe5a96d18b7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:19.764905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d8b60-c4d6-4e5a-95ca-fe5a96d18b7a service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:19.774168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 602/1198] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:06:22.106094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e594-8f89-4aa0-88f6-13e1a8340cdb None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:22.106629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e594-8f89-4aa0-88f6-13e1a8340cdb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:22.106629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e594-8f89-4aa0-88f6-13e1a8340cdb None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:22.107907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 597/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:22.113973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67516edc-67fb-45a2-b7de-8c6c6ef194d6 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:22.114422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67516edc-67fb-45a2-b7de-8c6c6ef194d6 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:22.114548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67516edc-67fb-45a2-b7de-8c6c6ef194d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:22.133573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-67516edc-67fb-45a2-b7de-8c6c6ef194d6 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:22.134757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-67516edc-67fb-45a2-b7de-8c6c6ef194d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:22.142493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 603/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:22.294926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e594-8f89-4aa0-88f6-13e1a8340cdb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:22.309439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e594-8f89-4aa0-88f6-13e1a8340cdb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:22.314928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47decfd-0926-4fc4-95de-adf95dae296b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:22.315777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47decfd-0926-4fc4-95de-adf95dae296b service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:22.316089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47decfd-0926-4fc4-95de-adf95dae296b service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:22.316364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47decfd-0926-4fc4-95de-adf95dae296b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:22.322336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:06:22.325114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47decfd-0926-4fc4-95de-adf95dae296b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:22.326665 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47decfd-0926-4fc4-95de-adf95dae296b service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:22.335897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 598/1201] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:23.398156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490880-a61d-4bf6-866e-924d9fdde10d None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:23.398721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490880-a61d-4bf6-866e-924d9fdde10d None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:23.398721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490880-a61d-4bf6-866e-924d9fdde10d None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:23.400082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 604/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 17:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:23.406845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f130f-d1c7-41cf-9c79-ee5a4f16f996 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:23.407275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f130f-d1c7-41cf-9c79-ee5a4f16f996 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:23.407381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f130f-d1c7-41cf-9c79-ee5a4f16f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:23.426468 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2e5f130f-d1c7-41cf-9c79-ee5a4f16f996 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:23.427649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2e5f130f-d1c7-41cf-9c79-ee5a4f16f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:23.435317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 599/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 17:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:23.587197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2490880-a61d-4bf6-866e-924d9fdde10d None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:23.601244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2490880-a61d-4bf6-866e-924d9fdde10d None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:23.607142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f93619-002d-443f-8dad-4d9aaa9661a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:23.607775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f93619-002d-443f-8dad-4d9aaa9661a4 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:23.608079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f93619-002d-443f-8dad-4d9aaa9661a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:23.608354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f93619-002d-443f-8dad-4d9aaa9661a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:23.616250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f93619-002d-443f-8dad-4d9aaa9661a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:23.617826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f93619-002d-443f-8dad-4d9aaa9661a4 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:23.627314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 605/1204] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:27.549635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ab27-075d-4340-80ae-65d95c2efe5a None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:27.550226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ab27-075d-4340-80ae-65d95c2efe5a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:27.550226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ab27-075d-4340-80ae-65d95c2efe5a None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:27.551526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 600/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:27.556791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f150485-715b-45e2-85e7-2ff22e34472b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:27.557197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f150485-715b-45e2-85e7-2ff22e34472b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:27.557330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f150485-715b-45e2-85e7-2ff22e34472b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:27.577538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0f150485-715b-45e2-85e7-2ff22e34472b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:27.578704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f150485-715b-45e2-85e7-2ff22e34472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:27.586347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 606/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:27.868596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40ab27-075d-4340-80ae-65d95c2efe5a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:27.870324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3e40ab27-075d-4340-80ae-65d95c2efe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:27.927344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40ab27-075d-4340-80ae-65d95c2efe5a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:27.930832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658d457-a65e-474b-b9c9-5420b1ff0592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:27.931438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d457-a65e-474b-b9c9-5420b1ff0592 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:27.931717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d457-a65e-474b-b9c9-5420b1ff0592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:27.931990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d457-a65e-474b-b9c9-5420b1ff0592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:27.941746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658d457-a65e-474b-b9c9-5420b1ff0592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:27.943155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658d457-a65e-474b-b9c9-5420b1ff0592 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:27.952391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 601/1207] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:30.069452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70fd9-f26d-4167-ab05-50938fad1a7f None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:30.069995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70fd9-f26d-4167-ab05-50938fad1a7f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:30.069995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70fd9-f26d-4167-ab05-50938fad1a7f None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:30.071265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 607/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:30.076996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e750f7-c304-4460-b87d-8d6d8d76a2b3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:30.077423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e750f7-c304-4460-b87d-8d6d8d76a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:30.077535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e750f7-c304-4460-b87d-8d6d8d76a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:30.097293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f1e750f7-c304-4460-b87d-8d6d8d76a2b3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:30.098482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f1e750f7-c304-4460-b87d-8d6d8d76a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:30.106304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 602/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:30.391528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70fd9-f26d-4167-ab05-50938fad1a7f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:30.402173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70fd9-f26d-4167-ab05-50938fad1a7f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:30.407774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b6a86-9f8f-4c37-9ff7-51cba4376178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:30.407774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b6a86-9f8f-4c37-9ff7-51cba4376178 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:30.407774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b6a86-9f8f-4c37-9ff7-51cba4376178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:30.407774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b6a86-9f8f-4c37-9ff7-51cba4376178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:30.415584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b6a86-9f8f-4c37-9ff7-51cba4376178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:30.417038 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b6a86-9f8f-4c37-9ff7-51cba4376178 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:30.426278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 608/1210] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:33.985193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0afcae-8b17-4b4b-8675-17dbce831fbd None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:33.985724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0afcae-8b17-4b4b-8675-17dbce831fbd None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:33.985724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0afcae-8b17-4b4b-8675-17dbce831fbd None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:33.987230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 603/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:33.992450 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2399517-9293-4074-9f6e-3acdbe7460fd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:33.992844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2399517-9293-4074-9f6e-3acdbe7460fd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:33.993015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2399517-9293-4074-9f6e-3acdbe7460fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:34.014737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c2399517-9293-4074-9f6e-3acdbe7460fd None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:34.015940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c2399517-9293-4074-9f6e-3acdbe7460fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:34.024077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 609/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:34.334025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0afcae-8b17-4b4b-8675-17dbce831fbd None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:34.339016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ea0afcae-8b17-4b4b-8675-17dbce831fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:34.394294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0afcae-8b17-4b4b-8675-17dbce831fbd None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:34.397819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ad00-4ded-4225-ae54-c1b79451920b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:34.398465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ad00-4ded-4225-ae54-c1b79451920b service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:34.398746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ad00-4ded-4225-ae54-c1b79451920b service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:34.399053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ad00-4ded-4225-ae54-c1b79451920b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:34.407902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2ad00-4ded-4225-ae54-c1b79451920b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:34.409460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2ad00-4ded-4225-ae54-c1b79451920b service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:34.418910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 604/1213] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:36.531074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2191-ecfc-4392-a15d-6e53c6122631 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:36.531672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2191-ecfc-4392-a15d-6e53c6122631 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:36.531672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2191-ecfc-4392-a15d-6e53c6122631 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:36.532983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 610/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 17:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:06:36.538285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05118-0569-450c-877a-569524b0bbd8 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:36.538725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05118-0569-450c-877a-569524b0bbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:36.538869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05118-0569-450c-877a-569524b0bbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:36.558601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-16c05118-0569-450c-877a-569524b0bbd8 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:06:36.559800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-16c05118-0569-450c-877a-569524b0bbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:36.567876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 605/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 17:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:06:36.877339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2191-ecfc-4392-a15d-6e53c6122631 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:06:36.879010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c07c2191-ecfc-4392-a15d-6e53c6122631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:06:36.886565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=61 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:06:36.892422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2191-ecfc-4392-a15d-6e53c6122631 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:06:36.895892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22c658-2018-465c-89fc-6b77b336a5d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:06:36.896516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c658-2018-465c-89fc-6b77b336a5d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:06:36.898795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c658-2018-465c-89fc-6b77b336a5d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:06:36.898795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c658-2018-465c-89fc-6b77b336a5d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:06:36.905870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22c658-2018-465c-89fc-6b77b336a5d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:06:36.907266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22c658-2018-465c-89fc-6b77b336a5d5 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:06:36.916819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 611/1216] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:06:46.553321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=63 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:06:46.918258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:00.310903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c678-bdc6-4bb9-b9b6-cd1e619ac93e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.314335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c678-bdc6-4bb9-b9b6-cd1e619ac93e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.314335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c678-bdc6-4bb9-b9b6-cd1e619ac93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.333983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e03b8-07a9-4868-ba19-d76e2975d8d2 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.334584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e03b8-07a9-4868-ba19-d76e2975d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.334584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e03b8-07a9-4868-ba19-d76e2975d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.352749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-9e97c678-bdc6-4bb9-b9b6-cd1e619ac93e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.354028 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9e97c678-bdc6-4bb9-b9b6-cd1e619ac93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.365862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 606/1217] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.369399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6b3e03b8-07a9-4868-ba19-d76e2975d8d2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.369399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6b3e03b8-07a9-4868-ba19-d76e2975d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.376548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb079-47ca-4c47-a696-a61ff092a6cc None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.376899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb079-47ca-4c47-a696-a61ff092a6cc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.377108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb079-47ca-4c47-a696-a61ff092a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.388566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 612/1218] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.389920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a444-d511-434c-aa5b-73dcd580cbd7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.390207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a444-d511-434c-aa5b-73dcd580cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.390481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a444-d511-434c-aa5b-73dcd580cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.417735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a36fb079-47ca-4c47-a696-a61ff092a6cc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.420435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a36fb079-47ca-4c47-a696-a61ff092a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.437835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 607/1219] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.439156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.439443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.439719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.441685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-76e9a444-d511-434c-aa5b-73dcd580cbd7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.442944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-76e9a444-d511-434c-aa5b-73dcd580cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.465004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 613/1220] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.466286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.466572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.466850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.473317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.474737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.492733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 608/1221] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.493578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.507592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.508821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.512392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504d22d-0539-4cad-81a9-56651817b622 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.521482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.522101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.522381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.522656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.523317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:00.523620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.524690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:08:00.524690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 17:08:00.524690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 17:08:00.525548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.529476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 614/1222] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.530121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.545941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eac1c3-5e35-4e7d-83cd-56c51c1f789e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.552719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 609/1223] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:00.554085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.561997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.562530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.562814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.563082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.563744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:00.564059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.565998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.575723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d00ec-06f4-4824-841d-76b17bc24406 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.595911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 615/1224] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:00.596799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.596974 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.597404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.597688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.598008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.598970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:00.599287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.601134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.607727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e998dbf-0246-451b-9c16-237f7d89e5f0 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.624725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.625362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.625646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.625923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.626573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.627674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:08:00.627674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 17:08:00.627674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 17:08:00.629259 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.630503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 610/1225] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:00.631671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.650114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6c3b2-d4e6-49b8-9677-5835646e471c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.662412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba388d4-d279-4f88-b5b4-eb841986c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.663311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba388d4-d279-4f88-b5b4-eb841986c92f admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.663598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba388d4-d279-4f88-b5b4-eb841986c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.663926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba388d4-d279-4f88-b5b4-eb841986c92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.664608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba388d4-d279-4f88-b5b4-eb841986c92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.666512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba388d4-d279-4f88-b5b4-eb841986c92f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.717870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 616/1226] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:00.718636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.768259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 611/1227] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/projects => generated 403 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:00.769619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.769914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.770202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.785610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea739cb8-6e72-46bd-9ee4-0221d210c7a7 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.791358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.793491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.849238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7927f16-8bc1-43b7-80b9-147b210709aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.849771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7927f16-8bc1-43b7-80b9-147b210709aa admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.850046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7927f16-8bc1-43b7-80b9-147b210709aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.850314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7927f16-8bc1-43b7-80b9-147b210709aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.850937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7927f16-8bc1-43b7-80b9-147b210709aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.852940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7927f16-8bc1-43b7-80b9-147b210709aa admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.860754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 612/1228] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:00.860933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:00.872352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e348dd-d89c-4357-8503-05ec8de6fde0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:00.877072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:00.878998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.878998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.878998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.878998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:00.879938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:08:00.879938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 17:08:00.879938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 17:08:00.881170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3284306-e4b7-4cda-85c6-0f58316ef223 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:00.954873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 617/1229] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/projects => generated 365 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:00.954873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85fdef-15ae-4088-b721-5408877143a0 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.954873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85fdef-15ae-4088-b721-5408877143a0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.954873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85fdef-15ae-4088-b721-5408877143a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.972288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6e85fdef-15ae-4088-b721-5408877143a0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.976106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 613/1230] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:00.976267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6e85fdef-15ae-4088-b721-5408877143a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:00.976357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:00.976357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:00.976357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:00.995666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:00.996897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.006553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:01.033472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 618/1231] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.034914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3491-6bd0-4351-b81e-94b1d9f75339 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.035208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3491-6bd0-4351-b81e-94b1d9f75339 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.035475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3491-6bd0-4351-b81e-94b1d9f75339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.057780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 614/1232] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.058788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.062633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0fdc3491-6bd0-4351-b81e-94b1d9f75339 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.064864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0fdc3491-6bd0-4351-b81e-94b1d9f75339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.075533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d632158-a3fd-426b-8f00-d82eaeaecbf1 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.075884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 619/1233] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.077150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905ebd-7a72-4817-9429-bedf19908ffc None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.077430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905ebd-7a72-4817-9429-bedf19908ffc None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.077709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7905ebd-7a72-4817-9429-bedf19908ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.080527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.081135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.081430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.081686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.082308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.084594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.098663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f7905ebd-7a72-4817-9429-bedf19908ffc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.107923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f7905ebd-7a72-4817-9429-bedf19908ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.108276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 620/1234] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.109430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.109710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.109979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.128303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.129445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.137077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 621/1235] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.137790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.146786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d0183-e784-483f-ab7c-4eee7dadb449 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.151809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c3afc-13fa-42fd-90c9-5694a3d8d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.152427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3afc-13fa-42fd-90c9-5694a3d8d754 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.152708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3afc-13fa-42fd-90c9-5694a3d8d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.152977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3afc-13fa-42fd-90c9-5694a3d8d754 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.153610 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c3afc-13fa-42fd-90c9-5694a3d8d754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.155509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c3afc-13fa-42fd-90c9-5694a3d8d754 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.185225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:01.191304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 615/1236] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/users => generated 302 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.192262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.201758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd260fd-826d-4ad6-b16b-5d860dddd58e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.206275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38ec1b-7f9c-4817-80ab-8f3f1cfc1ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.206839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38ec1b-7f9c-4817-80ab-8f3f1cfc1ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.207116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38ec1b-7f9c-4817-80ab-8f3f1cfc1ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.207384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38ec1b-7f9c-4817-80ab-8f3f1cfc1ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.207994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38ec1b-7f9c-4817-80ab-8f3f1cfc1ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.209599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38ec1b-7f9c-4817-80ab-8f3f1cfc1ce8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.220222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 616/1237] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:01.221376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fea16-2f90-488e-a81d-b6d976d50a85 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.221660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fea16-2f90-488e-a81d-b6d976d50a85 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.221937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fea16-2f90-488e-a81d-b6d976d50a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.240874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2f4fea16-2f90-488e-a81d-b6d976d50a85 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.242055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2f4fea16-2f90-488e-a81d-b6d976d50a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.248374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 622/1238] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.249647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee72d31-a593-47a7-b1f4-55823a092db4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.249933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee72d31-a593-47a7-b1f4-55823a092db4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.250206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee72d31-a593-47a7-b1f4-55823a092db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.252995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 617/1239] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.252995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.252995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.252995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.270746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8ee72d31-a593-47a7-b1f4-55823a092db4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.271976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8ee72d31-a593-47a7-b1f4-55823a092db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.274330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.275478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.280660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 623/1240] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.281876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.282157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.282427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.284529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 618/1241] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.285380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.295359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e86e5-0cd8-4ad7-bd36-cc6e3c04cdb4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.300275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.300867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.301152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.301417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.302057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:01.302357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.303918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.305485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:01.306659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:01.314701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 624/1242] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:01.315762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.323710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 619/1243] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:01.323710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.328280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aef4be-25e1-4265-9b3b-05e9c29ecaaa None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.329362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653e7d-99ed-47a8-a2a6-49c1e963f341 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.332801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.333390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.333666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.333828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.334404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.334568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.334990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.334990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.335217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.335546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.336927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.337117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.347348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 625/1244] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:01.348130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 620/1245] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:01.349025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.351118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.360318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965d06-aa52-4af5-8514-d83110952784 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.361051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef08aa-1a1d-438a-9671-f0050cda10f5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.364891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.365474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.365642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.365784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.366011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.366215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.366498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.366767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:01.366885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.367073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.367425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.368837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.369007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.378837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 626/1246] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:01.379634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.381197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 621/1247] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:01.381986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.392385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9988d-8843-4bc8-971d-6bbf2bcf1f27 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.393806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9fa6-2565-45ae-8056-4fe3da110b62 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.396986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.397568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.397838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.398098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.398343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.398730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:01.398938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.399045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.399208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.399475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.400137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:01.400436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.400627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.402624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.414011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 622/1248] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:01.414825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.414951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 627/1249] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:01.415539 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.424500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d41ea-aed6-4447-897b-c5d92e7db51d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.429304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.429917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.430192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.430460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.431086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:01.431381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.431709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ae0b-6ce7-4994-8142-ea874bc11667 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.433006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.436402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.436989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.437269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.437538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.438155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.440001 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.446300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 623/1250] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:01.447103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.457291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5108-2f54-4b7a-9498-083b1e96c688 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.462192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.462772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.463071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.463338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.464632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.466186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.475747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 624/1251] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:01.476529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.485585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf725a-0c25-4d78-a1df-57e2dac234c5 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.490276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.490893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.491174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.491449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] PATH_INFO: `/v3/projects/f9c633053a92460c8fd6858e66679fd3/users/f6d39546d0504953a4f9d769db9421a8/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.505683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c633053a92460c8fd6858e66679fd3, user_id=f6d39546d0504953a4f9d769db9421a8, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.509613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.560813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 628/1252] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.560813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.579660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 625/1253] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:01 2022] PUT /identity/v3/projects/f9c633053a92460c8fd6858e66679fd3/users/f6d39546d0504953a4f9d769db9421a8/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:01.580679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.620520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fbb26-71b3-4db1-8df9-bf64d65a3d1e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.643147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1062a-ff73-4c45-a8dc-a30ca4cfbc6f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.647919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.648570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.648853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.649128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.649764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.652397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.654399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:01.677031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.677646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.677925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.678202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.678851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.680455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.690046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 629/1254] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:01.691068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.701134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483e197-507e-4008-bec9-5e3bb5b63bcf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.705993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.706559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.706839 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.707117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.707710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.709596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.740736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 626/1255] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.741659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.762634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:01.797739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 630/1256] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.798527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.802977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d075-e707-4df6-befb-4291e31f6e86 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.862104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.862385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66babd58-1ba9-467e-8985-651784360d60 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:01.862784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.863176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.863587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.864363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.866273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.873769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:01.874337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:01.874624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:01.874910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:01.875553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:01.877984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:01.949775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 627/1257] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/projects => generated 389 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.949775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:01.968548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 631/1258] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:01.969360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.009528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b305-2d3d-4e17-b91f-7501c12b8aef admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.032458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e1e51-901a-40de-8641-8a57556d009a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.037052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81684d83-ffe6-4a71-b843-f5dd04b95b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.037670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81684d83-ffe6-4a71-b843-f5dd04b95b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.037931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81684d83-ffe6-4a71-b843-f5dd04b95b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.038198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81684d83-ffe6-4a71-b843-f5dd04b95b7a admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.038823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81684d83-ffe6-4a71-b843-f5dd04b95b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.040764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81684d83-ffe6-4a71-b843-f5dd04b95b7a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.067123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.067787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.068094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.068373 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] PATH_INFO: `/v3/projects/f10a89146d6345388ca2d5d390c472ea/users/2f0a9facc2b64d2ba4bcf899d7823b86/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.070655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:02.083366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10a89146d6345388ca2d5d390c472ea, user_id=2f0a9facc2b64d2ba4bcf899d7823b86, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.086215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.127913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 632/1259] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:01 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:02.129248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.129534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.129812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.148941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:02.155777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:02.155896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 628/1260] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:01 2022] PUT /identity/v3/projects/f10a89146d6345388ca2d5d390c472ea/users/2f0a9facc2b64d2ba4bcf899d7823b86/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:02.155896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.170759 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:02.209479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 633/1261] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:02.210291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.218774 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda50a40-4b33-4b99-898e-1caef3e01337 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.222296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd065a-bf60-4161-86cc-8eb263de2c9b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.223867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.224553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.224833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.225227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] PATH_INFO: `/v3/projects/dc5beb1195b343f2b0f9ac6d20d4b0f8/users/b8d24868c409409fab54ca4cfcbf86ea/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.228287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.228897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.229681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.229958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.230587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.232557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.240990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5beb1195b343f2b0f9ac6d20d4b0f8, user_id=b8d24868c409409fab54ca4cfcbf86ea, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.243664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.333133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 634/1262] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:02.334070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.343313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ce5a-eadc-4d6f-947c-8b92dd502808 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.347850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.348451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.348728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.348997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.349606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.351499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.357039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 629/1263] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:02 2022] PUT /identity/v3/projects/dc5beb1195b343f2b0f9ac6d20d4b0f8/users/b8d24868c409409fab54ca4cfcbf86ea/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:02.357643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.425443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04484683-599c-49aa-b53c-e393f131bd07 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.430152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.430816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 635/1264] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:02.430951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.431095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.431487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.431581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.432166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.434145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.443719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beed77-3de6-493d-9c04-a34a8e0320ce admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.449016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d39e1-b7c2-4e24-84b9-18e55c20fbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.449602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d39e1-b7c2-4e24-84b9-18e55c20fbfc admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.449881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d39e1-b7c2-4e24-84b9-18e55c20fbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.450152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d39e1-b7c2-4e24-84b9-18e55c20fbfc admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.450801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d39e1-b7c2-4e24-84b9-18e55c20fbfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.452830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d39e1-b7c2-4e24-84b9-18e55c20fbfc admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.516082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 630/1265] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:02.516910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.526771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e95098-9e06-4d5f-8b32-74281b8a77d3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.531306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.531966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.532287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.532560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.533192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.534756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.539329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 636/1266] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:02.542622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.542622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.542622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.548959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 631/1267] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.548959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.561989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:02.563196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:02.571371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 637/1268] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:02.572378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.581779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9c752-4a54-4679-bf1c-0f6cce1855bb None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.586368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcced63a-1396-4395-b65a-2293971362fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.586971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcced63a-1396-4395-b65a-2293971362fd admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.589261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcced63a-1396-4395-b65a-2293971362fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.589261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcced63a-1396-4395-b65a-2293971362fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.589261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcced63a-1396-4395-b65a-2293971362fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.590329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcced63a-1396-4395-b65a-2293971362fd admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.593110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:02.603274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 638/1269] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.603274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.603274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.603274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.603697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8e529-515b-491a-b514-2c5c11d1b048 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.606956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.607577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.607851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.608149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.617392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.618854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.628318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 632/1270] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:02.629155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.631427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] MFA Rules not processed for user `b8d24868c409409fab54ca4cfcbf86ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:02.635513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:02.638549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb3f2-21bb-4968-b7e5-f57f7b65997f service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.644144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.644746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.645177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.645460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.646474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.648131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.661218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 633/1271] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.664978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.686205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ecdb8-cbd0-45a1-9999-8d5a19177756 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.686205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.686205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.686205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.686669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.686669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.686669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.696701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 634/1272] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.697586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.705835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 639/1273] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:02.706926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.709654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007213d-8e5b-4613-934c-3aad74093fd1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.714870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.715466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.715746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.716019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.716673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.717509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5f7eb-2139-43ce-bc05-a0e8d381e58e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.718248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.722080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.722672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.722953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.723218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.723849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.725454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.728418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 635/1274] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.729868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.737209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 640/1275] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.738164 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.740305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d04e93-a6f0-48f1-bb26-8ecd8c205c64 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.744190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.744833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.745232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.745648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.747431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7133-e0ff-4195-b6eb-54b76f8ba13a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.751932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.752516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.752798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.753063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.753679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.755124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.755253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.756872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.765225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 641/1276] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.765890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.772138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 636/1277] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:02.772138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.777720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256ae8a-35b0-4374-967f-2d28bcdc97e3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.779558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846f6c2-114b-4355-820d-34f0c2c6af61 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.782623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.783278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.783565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.783823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.784493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.784667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.785355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.785677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.786017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.786128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.786790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.788441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.797341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 642/1278] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.799294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 637/1279] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.800968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.801097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.811749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00ac23-6d08-4d8a-9f24-d46f162333ad admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.814296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19df4-9e33-4ef2-aa88-5bc5f22e6836 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.816717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.817416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.817623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.817969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.818534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.820142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.820925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.821602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.821934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.822282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.824082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.826150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.832614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 638/1280] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:02.836903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.837822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:02.848231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16767d91-941c-455e-9c4f-fee57bc09ce6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:02.852965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:02.853695 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:02.854036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:02.854407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] PATH_INFO: `/v3/projects/8b522f19433f4c6880f47901244f5da4/users/8e7dd537e9cc46809acd60699788049c/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:02.874599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b522f19433f4c6880f47901244f5da4, user_id=8e7dd537e9cc46809acd60699788049c, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:02.877541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:02.921924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 643/1281] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:02 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:02.921924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.946111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 639/1282] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:02 2022] PUT /identity/v3/projects/8b522f19433f4c6880f47901244f5da4/users/8e7dd537e9cc46809acd60699788049c/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:02.946111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:02.994396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ff1c-76a3-4405-83c6-821457d5321c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.018841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0e4a6-aaf6-4c70-b3a9-6bb71d7ec93b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.023731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.024512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.024967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.025638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] PATH_INFO: `/v3/projects/3f5f061a8d22484faf392b34320f17b3/users/d630884096ad491bbf26959cdb0f28a2/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.044213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5f061a8d22484faf392b34320f17b3, user_id=d630884096ad491bbf26959cdb0f28a2, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.048616 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.066077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.067064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.067463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.067863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] PATH_INFO: `/v3/projects/f3ec355ce8444446afea97b72da044a4/users/5e09aafc4a9644d4acc2303fff71913f/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.089226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ec355ce8444446afea97b72da044a4, user_id=5e09aafc4a9644d4acc2303fff71913f, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.091944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.116255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 640/1283] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:02 2022] PUT /identity/v3/projects/3f5f061a8d22484faf392b34320f17b3/users/d630884096ad491bbf26959cdb0f28a2/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:03.117306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.159486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:03.184471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740523b1-7d7d-431e-861c-e71e30fb68d0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.185613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 644/1284] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:02 2022] PUT /identity/v3/projects/f3ec355ce8444446afea97b72da044a4/users/5e09aafc4a9644d4acc2303fff71913f/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:03.197870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.253092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.254093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.254393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.254852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] PATH_INFO: `/v3/projects/34f584f725354f68b917b45b645b3630/users/b8507de448b6455fb792bda0c949421d/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.271662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f584f725354f68b917b45b645b3630, user_id=b8507de448b6455fb792bda0c949421d, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.271935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff659ede-f0d7-44c5-9899-68249b85f849 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.274904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.277649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd6e46-d92e-4998-a065-55e78b397e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.278319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd6e46-d92e-4998-a065-55e78b397e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.278605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd6e46-d92e-4998-a065-55e78b397e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.278886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd6e46-d92e-4998-a065-55e78b397e80 admin admin] PATH_INFO: `/v3/projects/18aef1375628417b91f816bccd05d1cf/users/8bb27ad53d4a454eb10268ab27ad3258/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.293479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd6e46-d92e-4998-a065-55e78b397e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18aef1375628417b91f816bccd05d1cf, user_id=8bb27ad53d4a454eb10268ab27ad3258, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.296138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd6e46-d92e-4998-a065-55e78b397e80 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.380679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 641/1285] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:03 2022] PUT /identity/v3/projects/34f584f725354f68b917b45b645b3630/users/b8507de448b6455fb792bda0c949421d/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:03.381507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.402903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 645/1286] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:03 2022] PUT /identity/v3/projects/18aef1375628417b91f816bccd05d1cf/users/8bb27ad53d4a454eb10268ab27ad3258/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:03.406121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f4181-d468-435b-99ec-7f34dca8820e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.406121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f4181-d468-435b-99ec-7f34dca8820e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.406121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f4181-d468-435b-99ec-7f34dca8820e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.415916 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:03.438825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9a2f4181-d468-435b-99ec-7f34dca8820e None None] MFA Rules not processed for user `8e7dd537e9cc46809acd60699788049c`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:03.440246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9a2f4181-d468-435b-99ec-7f34dca8820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:03.473970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d99e0-6fab-4b45-81b1-6750fa0280b6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.526810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 646/1287] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:03.527863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97d29-d828-41b4-a6bb-9822e135a567 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.528184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97d29-d828-41b4-a6bb-9822e135a567 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.528456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97d29-d828-41b4-a6bb-9822e135a567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.559365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd90e19-5c42-4834-8e64-279dc91586c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.560923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90e19-5c42-4834-8e64-279dc91586c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.564324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90e19-5c42-4834-8e64-279dc91586c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.564805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90e19-5c42-4834-8e64-279dc91586c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.566497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd90e19-5c42-4834-8e64-279dc91586c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.571315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd90e19-5c42-4834-8e64-279dc91586c9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.571483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-94a97d29-d828-41b4-a6bb-9822e135a567 None None] MFA Rules not processed for user `d630884096ad491bbf26959cdb0f28a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:03.572768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-94a97d29-d828-41b4-a6bb-9822e135a567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:03.641984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 647/1288] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:03.644062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.644062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.644062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.657224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 642/1289] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:03.658271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea18bb-f6d0-4c94-b6b2-64a20154715e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.658713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea18bb-f6d0-4c94-b6b2-64a20154715e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.659725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea18bb-f6d0-4c94-b6b2-64a20154715e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.669262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:03.685712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-27ea18bb-f6d0-4c94-b6b2-64a20154715e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:03.685961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] MFA Rules not processed for user `5e09aafc4a9644d4acc2303fff71913f`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:03.687210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:03.687942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-27ea18bb-f6d0-4c94-b6b2-64a20154715e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:03.698446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 643/1290] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:03.700702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.707105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.707518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.771115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 648/1291] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:03.772077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.785440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b1af2-090a-413e-9e94-6a9ae88adf27 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.790893 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e3415-e9dc-4558-91b7-6c11cd4bbab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.792293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e3415-e9dc-4558-91b7-6c11cd4bbab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.792729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e3415-e9dc-4558-91b7-6c11cd4bbab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.793100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e3415-e9dc-4558-91b7-6c11cd4bbab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.793944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e3415-e9dc-4558-91b7-6c11cd4bbab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.796679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e3415-e9dc-4558-91b7-6c11cd4bbab2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.813093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:03.856436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:03.857726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:03.866705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 644/1292] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:03.867553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.878823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c659c0-18af-44ea-94ad-566b6b7f6d00 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.884095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1695-3683-43a3-9312-9eee28696213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:03.884709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1695-3683-43a3-9312-9eee28696213 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.884982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1695-3683-43a3-9312-9eee28696213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.885249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1695-3683-43a3-9312-9eee28696213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.886018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1695-3683-43a3-9312-9eee28696213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:03.887755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1695-3683-43a3-9312-9eee28696213 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:03.899165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 645/1293] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:03.905033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8fbd-ac56-4dbb-9843-152bed714bf9 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.905033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8fbd-ac56-4dbb-9843-152bed714bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.905033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8fbd-ac56-4dbb-9843-152bed714bf9 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.905033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 646/1294] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 10 17:08:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:03.905033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc8fbd-ac56-4dbb-9843-152bed714bf9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.931404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 649/1295] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:03 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:03.933746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed87c09-b266-4bcc-a366-bc4595e75385 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:03.934276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed87c09-b266-4bcc-a366-bc4595e75385 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:03.934711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed87c09-b266-4bcc-a366-bc4595e75385 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:03.936403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 650/1296] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 10 17:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:03.938100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed87c09-b266-4bcc-a366-bc4595e75385 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:03.969706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc8fbd-ac56-4dbb-9843-152bed714bf9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:03.996799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed87c09-b266-4bcc-a366-bc4595e75385 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.001493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.002122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.002420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.002680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.024508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76164814-e7a5-47ff-b1f2-42710fa0fc7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.025143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76164814-e7a5-47ff-b1f2-42710fa0fc7a service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.025512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76164814-e7a5-47ff-b1f2-42710fa0fc7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.025660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76164814-e7a5-47ff-b1f2-42710fa0fc7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.085698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.086927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.092313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76164814-e7a5-47ff-b1f2-42710fa0fc7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.093738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76164814-e7a5-47ff-b1f2-42710fa0fc7a service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.098491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 651/1297] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:04.101700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.109292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4b469-c520-453e-b964-7e64efae6019 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.111091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 647/1298] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:04.114142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.115124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.115409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.116058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] PATH_INFO: `/v3/projects/27db1ca730c74d22993885d0d5c4c353/users/be12872accd94680834de30ad50db593/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.121811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8520-5566-4055-8cd3-1905aed4f62a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.122239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8520-5566-4055-8cd3-1905aed4f62a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.122527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8520-5566-4055-8cd3-1905aed4f62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.132714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27db1ca730c74d22993885d0d5c4c353, user_id=be12872accd94680834de30ad50db593, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.135554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.169801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-113c8520-5566-4055-8cd3-1905aed4f62a None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:04.171049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-113c8520-5566-4055-8cd3-1905aed4f62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:04.199406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:04.210202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 652/1299] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:04 2022] PUT /identity/v3/projects/27db1ca730c74d22993885d0d5c4c353/users/be12872accd94680834de30ad50db593/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:04.214472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.242080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 648/1300] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 10 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:04.243918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.244478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.245125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.270572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:04.273011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:04.282514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98e32e-4783-4cf8-8956-880c52297921 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.287938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.288668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.288998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.289342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.290041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.292986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.378515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 649/1301] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 10 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:04.379372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.391936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 653/1302] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:04 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:04.392458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e7382-7926-4e01-9bc7-9a79546a7dfa None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.392852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.399529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.400324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.401828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.401828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.402284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.404099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.410437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d823ed-d5d8-4ed2-9bc6-7e8defc4f99c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.416925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 650/1303] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:04.418593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.428694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.429482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.429839 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.430184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.439398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:04.445714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84a510-33bc-4670-b7c8-977bb3f05f4f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.455270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a38a6-dd43-4610-9ad1-eb812de8f942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.456233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a38a6-dd43-4610-9ad1-eb812de8f942 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.456644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a38a6-dd43-4610-9ad1-eb812de8f942 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.456937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a38a6-dd43-4610-9ad1-eb812de8f942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.509899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.517546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.531707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 654/1304] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:04.533585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.547398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bbaac-7b22-41bf-8ed1-81aa9cc273d6 service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.553331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.555801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.556478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.557305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.558039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.559899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.575287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 655/1305] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:04.577164 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.594788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384a0c5-7a2e-44eb-ac6c-04feff4954c6 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.601938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.602867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.603457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.605466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] PATH_INFO: `/v3/projects/27db1ca730c74d22993885d0d5c4c353/users/be12872accd94680834de30ad50db593/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.609425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27db1ca730c74d22993885d0d5c4c353, user_id=be12872accd94680834de30ad50db593, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.614072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.626299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a38a6-dd43-4610-9ad1-eb812de8f942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.628228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a38a6-dd43-4610-9ad1-eb812de8f942 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.647248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 651/1306] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:04.647347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 656/1307] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:04 2022] PUT /identity/v3/projects/27db1ca730c74d22993885d0d5c4c353/users/be12872accd94680834de30ad50db593/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:04.647347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:04.649087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eceb11-4129-4235-914e-74eb1cc3acda None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.649572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eceb11-4129-4235-914e-74eb1cc3acda None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.650235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eceb11-4129-4235-914e-74eb1cc3acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.690306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-58eceb11-4129-4235-914e-74eb1cc3acda None None] MFA Rules not processed for user `2f0a9facc2b64d2ba4bcf899d7823b86`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:04.692458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-58eceb11-4129-4235-914e-74eb1cc3acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:04.744241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:04.764315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826bad-c8cb-4194-95f7-4cc388088524 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:04.787490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:04.787490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.787490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.787490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] PATH_INFO: `/v3/projects/80bb962108a94de28c806813c8debb57/users/274a6d0dae514d4ea0da4fb0e099c558/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.791631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 652/1308] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:04.793376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:04.793928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:04.794722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:04.822408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bb962108a94de28c806813c8debb57, user_id=274a6d0dae514d4ea0da4fb0e099c558, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:04.825781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:04.842043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] MFA Rules not processed for user `be12872accd94680834de30ad50db593`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:04.844079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:04.925115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 657/1309] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:04 2022] PUT /identity/v3/projects/80bb962108a94de28c806813c8debb57/users/274a6d0dae514d4ea0da4fb0e099c558/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:04.925330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.006929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 653/1310] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:05.008093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.038487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb372f0-0df7-4230-9e3d-69032d1a149c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.098472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d2687-8b6f-44e1-95e7-6b503b7db2b9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.109105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363fd05-2fef-405e-852f-a7d58641cca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.109741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363fd05-2fef-405e-852f-a7d58641cca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.110758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363fd05-2fef-405e-852f-a7d58641cca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.112206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363fd05-2fef-405e-852f-a7d58641cca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.113926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363fd05-2fef-405e-852f-a7d58641cca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.118024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363fd05-2fef-405e-852f-a7d58641cca8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.132574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.137974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.138194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.138319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.138959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.140875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.149446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:05.157625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 654/1311] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:05.158899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.241155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa4a45-8010-4d3c-af90-c21aaaac068e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.248928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 658/1312] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:04 2022] POST /identity/v3/projects => generated 403 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:05.250490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8aed-b910-495e-b6b6-6c91965953b0 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.250786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8aed-b910-495e-b6b6-6c91965953b0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.251065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8aed-b910-495e-b6b6-6c91965953b0 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.252516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 659/1313] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 10 17:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:05.253764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11d269-e503-437b-828d-e9143151961c None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.254048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11d269-e503-437b-828d-e9143151961c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.254355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11d269-e503-437b-828d-e9143151961c None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.257597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 660/1314] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 10 17:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:05.259989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11d269-e503-437b-828d-e9143151961c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.266381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:05.366702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dcb89-3058-4474-9593-30c081f934d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.367585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dcb89-3058-4474-9593-30c081f934d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.368061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dcb89-3058-4474-9593-30c081f934d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.368497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dcb89-3058-4474-9593-30c081f934d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.387354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11d269-e503-437b-828d-e9143151961c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.393105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.394046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.394431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.395034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] PATH_INFO: `/v3/projects/80bb962108a94de28c806813c8debb57/users/274a6d0dae514d4ea0da4fb0e099c558/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.397266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bb962108a94de28c806813c8debb57, user_id=274a6d0dae514d4ea0da4fb0e099c558, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.400758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.425526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 661/1315] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:05 2022] PUT /identity/v3/projects/80bb962108a94de28c806813c8debb57/users/274a6d0dae514d4ea0da4fb0e099c558/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:05.425526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.475607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3dcb89-3058-4474-9593-30c081f934d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.477973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3dcb89-3058-4474-9593-30c081f934d6 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.494392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 655/1316] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:05.497268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a821299-a5f3-444e-aea3-27dca613f692 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.498057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a821299-a5f3-444e-aea3-27dca613f692 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.498502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a821299-a5f3-444e-aea3-27dca613f692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.522654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7a821299-a5f3-444e-aea3-27dca613f692 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:05.525126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7a821299-a5f3-444e-aea3-27dca613f692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:05.539081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091293d-3194-4bb1-904d-4784d2e65a27 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.545425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9ca62-bbc6-4554-a96f-3f2cc2192b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.546217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9ca62-bbc6-4554-a96f-3f2cc2192b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.546506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9ca62-bbc6-4554-a96f-3f2cc2192b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.546780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9ca62-bbc6-4554-a96f-3f2cc2192b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.547423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9ca62-bbc6-4554-a96f-3f2cc2192b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.549374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9ca62-bbc6-4554-a96f-3f2cc2192b5e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.654980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 656/1317] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:08:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:05.656604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.656889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.657209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.673372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 662/1318] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:05 2022] POST /identity/v3/users => generated 308 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:05.674833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.675125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.675408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.712762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:05.715011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:05.727843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 657/1319] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:08:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:05.730633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.743878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] MFA Rules not processed for user `274a6d0dae514d4ea0da4fb0e099c558`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:05.745282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:05.798624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:05.805610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce3f24-d109-43f6-8344-81d2e3347e01 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.810572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.811504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.812001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.812458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.828280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.830767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.850742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 658/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:05.850742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.871363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd371be1-c2fb-4648-b72f-a4e22389c073 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.878681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.879526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.879954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.880513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.881482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:05.883761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:05.900286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 659/1321] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:05.902374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.915465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bd6f0-ac6a-41fa-a9a0-ebbefe87d6e2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.921455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.922418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.922702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.923105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:05.931437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:05.939887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 663/1322] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:05.941031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:05.968650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6c274-779e-4c72-b865-f61e1c3c0923 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:05.990048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:05.990829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:05.991012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:05.991291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] PATH_INFO: `/v3/projects/3ecb448ed11f4b42a24e97cb1976ba28/users/12ad4c60384c4a0bb8954f69936d2510/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:06.004339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:06.008618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:06.009388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecb448ed11f4b42a24e97cb1976ba28, user_id=12ad4c60384c4a0bb8954f69936d2510, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:06.013898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:06.030831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 660/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:06.100971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 664/1324] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:05 2022] PUT /identity/v3/projects/3ecb448ed11f4b42a24e97cb1976ba28/users/12ad4c60384c4a0bb8954f69936d2510/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:06.103385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:06.119800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb356e0-3d00-453d-9329-a33a95420fff service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:06.235897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:06.236826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:06.237347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:06.237832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:06.238750 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:06.240603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:06.269209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 661/1325] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:06.273682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:06.290119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a673a-43f9-4fd0-bea7-5261a4fd248a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:06.296392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:06.297344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:06.297497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:06.297777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] PATH_INFO: `/v3/projects/3ecb448ed11f4b42a24e97cb1976ba28/users/12ad4c60384c4a0bb8954f69936d2510/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:06.299790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecb448ed11f4b42a24e97cb1976ba28, user_id=12ad4c60384c4a0bb8954f69936d2510, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:06.302549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:06.314220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 665/1326] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:06 2022] PUT /identity/v3/projects/3ecb448ed11f4b42a24e97cb1976ba28/users/12ad4c60384c4a0bb8954f69936d2510/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:06.749602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:06.811409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9b59b-1b00-4382-89ef-de5c39bc44ba admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:06.853664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:06.910312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272b0b-e183-4c8e-9d7a-1365905aa8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:06.912572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272b0b-e183-4c8e-9d7a-1365905aa8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:06.912873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272b0b-e183-4c8e-9d7a-1365905aa8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:06.913160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272b0b-e183-4c8e-9d7a-1365905aa8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:07.040583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272b0b-e183-4c8e-9d7a-1365905aa8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:07.043658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272b0b-e183-4c8e-9d7a-1365905aa8f8 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:07.064410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 662/1327] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:07.199695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:07.214754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4810-2292-4be2-85a3-cdac5d1524f4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:07.223265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:07.224475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:07.224475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:07.224937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:07.369690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:07.379965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:07.395657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 666/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:10.123735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb19d-fb89-451c-9b76-f2c24b08a492 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:10.126035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb19d-fb89-451c-9b76-f2c24b08a492 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:10.126035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb19d-fb89-451c-9b76-f2c24b08a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:10.181076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-376bb19d-fb89-451c-9b76-f2c24b08a492 None None] MFA Rules not processed for user `12ad4c60384c4a0bb8954f69936d2510`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:10.181076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-376bb19d-fb89-451c-9b76-f2c24b08a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:10.198657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:10.345754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ad047-e42b-4f10-a0fa-994148029255 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:10.355638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:10.355638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:10.355638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:10.355638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] PATH_INFO: `/v3/users/8e7dd537e9cc46809acd60699788049c` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:10.355967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7dd537e9cc46809acd60699788049c)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:10.357432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: WARNING py.warnings [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:08:10.357432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: warnings.warn(msg) Oct 10 17:08:10.357432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: Oct 10 17:08:10.359764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:10.449251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 663/1329] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:10.658205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7dd537e9cc46809acd60699788049c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:10.802054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7dd537e9cc46809acd60699788049c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:10.802750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7dd537e9cc46809acd60699788049c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:10.803445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7dd537e9cc46809acd60699788049c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:10.809243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:10.872055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7dd537e9cc46809acd60699788049c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:10.916131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 667/1330] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:10 2022] DELETE /identity/v3/users/8e7dd537e9cc46809acd60699788049c => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:10.918217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:10.918511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:10.918793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:10.989035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:10.990305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:11.116242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:11.141574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 664/1331] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:11.155609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:11.157793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:11.308376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50eaf2-44b4-4304-a481-97ce254efa23 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:11.317454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661374a8-fc7f-4e7e-8a9b-8fa42b8b9d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:11.318944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661374a8-fc7f-4e7e-8a9b-8fa42b8b9d2c service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:11.319382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661374a8-fc7f-4e7e-8a9b-8fa42b8b9d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:11.323537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661374a8-fc7f-4e7e-8a9b-8fa42b8b9d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:11.333977 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661374a8-fc7f-4e7e-8a9b-8fa42b8b9d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:11.336572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661374a8-fc7f-4e7e-8a9b-8fa42b8b9d2c service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:11.351001 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 668/1332] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:11.435724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:11.435724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:11.472922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c1a17-1c56-497c-8c48-7f0e9d80c37d None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:11.488588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:11.490420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:11.490827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:11.491179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] PATH_INFO: `/v3/users/d630884096ad491bbf26959cdb0f28a2` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:11.494218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d630884096ad491bbf26959cdb0f28a2)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:11.501152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:11.898950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd630884096ad491bbf26959cdb0f28a2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:11.900398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd630884096ad491bbf26959cdb0f28a2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:12.007841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd630884096ad491bbf26959cdb0f28a2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:12.041306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd630884096ad491bbf26959cdb0f28a2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:12.042708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd630884096ad491bbf26959cdb0f28a2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:12.066944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 665/1333] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:11 2022] DELETE /identity/v3/users/d630884096ad491bbf26959cdb0f28a2 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:12.071356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1b665-e130-49e9-83d4-08fab964857c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:12.072101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1b665-e130-49e9-83d4-08fab964857c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:12.072558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1b665-e130-49e9-83d4-08fab964857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:12.093145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-12e1b665-e130-49e9-83d4-08fab964857c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:12.094789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-12e1b665-e130-49e9-83d4-08fab964857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:12.191122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 669/1334] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:12.228585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:12.228585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:12.394151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ec6d3-1dc1-4ebe-a709-a8df619df42a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:12.420728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d1aae-5c50-4543-b908-59781426ef4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:12.421368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d1aae-5c50-4543-b908-59781426ef4e service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:12.421650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d1aae-5c50-4543-b908-59781426ef4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:12.421923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d1aae-5c50-4543-b908-59781426ef4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:12.448802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d1aae-5c50-4543-b908-59781426ef4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:12.451357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d1aae-5c50-4543-b908-59781426ef4e service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:12.467559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 666/1335] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:12.750456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a274db-1b61-4455-9b8f-6e3c040d4ee8 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:12.751073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a274db-1b61-4455-9b8f-6e3c040d4ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:12.751237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a274db-1b61-4455-9b8f-6e3c040d4ee8 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:12.753054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 670/1336] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:12.786542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:12.787084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:12.787528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:12.825551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:12.831073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:12.987453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 667/1337] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:08:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:13.017158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a274db-1b61-4455-9b8f-6e3c040d4ee8 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:13.019020 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-75a274db-1b61-4455-9b8f-6e3c040d4ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:13.061413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a274db-1b61-4455-9b8f-6e3c040d4ee8 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:13.075955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:13.081035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:13.081035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:13.081035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:13.095406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:13.097607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:13.124169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 671/1338] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:13.844899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:13.847394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:13.873742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffcc45-e909-484c-905c-cc0477ae729c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:13.889034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:13.889911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:13.890355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:13.891905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] PATH_INFO: `/v3/projects/8b522f19433f4c6880f47901244f5da4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:13.893639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b522f19433f4c6880f47901244f5da4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:13.894669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: WARNING py.warnings [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 17:08:13.894669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: warnings.warn(msg) Oct 10 17:08:13.894669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: Oct 10 17:08:13.903805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:13.908254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:14.133214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b522f19433f4c6880f47901244f5da4', 'request_id': 'req-a5105521-fc89-44a1-96a0-2526f3fe90b2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.219266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b522f19433f4c6880f47901244f5da4', 'request_id': 'req-a5105521-fc89-44a1-96a0-2526f3fe90b2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.226595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b522f19433f4c6880f47901244f5da4', 'request_id': 'req-a5105521-fc89-44a1-96a0-2526f3fe90b2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.271168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] The token cache is being invalidate because project 8b522f19433f4c6880f47901244f5da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:14.277599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.278503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.281312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 668/1339] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:13 2022] DELETE /identity/v3/projects/8b522f19433f4c6880f47901244f5da4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:14.285503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:14.297468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44db7e-0815-4993-bbd4-6164608da312 service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:14.376403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:14.380856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:14.380856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:14.380856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] PATH_INFO: `/v3/users/274a6d0dae514d4ea0da4fb0e099c558` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:14.380856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=274a6d0dae514d4ea0da4fb0e099c558)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:14.384593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:14.393312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:14.590432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274a6d0dae514d4ea0da4fb0e099c558'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.616580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:14.621088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:14.650398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5105521-fc89-44a1-96a0-2526f3fe90b2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:14.664804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:14.665623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:14.665864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:14.666133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] PATH_INFO: `/v3/projects/3f5f061a8d22484faf392b34320f17b3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:14.668076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5f061a8d22484faf392b34320f17b3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:14.670810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:14.684067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274a6d0dae514d4ea0da4fb0e099c558'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.685027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274a6d0dae514d4ea0da4fb0e099c558'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.686565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274a6d0dae514d4ea0da4fb0e099c558'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.727645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-7f66491d-151f-420c-b9df-08a6d555e014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274a6d0dae514d4ea0da4fb0e099c558'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.777518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 672/1340] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:14 2022] DELETE /identity/v3/users/274a6d0dae514d4ea0da4fb0e099c558 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:14.869231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5f061a8d22484faf392b34320f17b3', 'request_id': 'req-ba3904bc-839d-4745-a28a-df625e111d5b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.930250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5f061a8d22484faf392b34320f17b3', 'request_id': 'req-ba3904bc-839d-4745-a28a-df625e111d5b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.947320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5f061a8d22484faf392b34320f17b3', 'request_id': 'req-ba3904bc-839d-4745-a28a-df625e111d5b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.973394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] The token cache is being invalidate because project 3f5f061a8d22484faf392b34320f17b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:14.974128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.978016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba3904bc-839d-4745-a28a-df625e111d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:14.983974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 669/1341] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:14 2022] DELETE /identity/v3/projects/3f5f061a8d22484faf392b34320f17b3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:14.989356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:14.994115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:14.994115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.024600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:15.026490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:15.163458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 673/1342] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:15.165186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.165748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.166759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.216400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:15.217680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:15.245116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 670/1343] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:15.249689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:15.261876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee805f-d1f2-4f1e-9dc5-4defe839c09d None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:15.268017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:15.268465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.268876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.269399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.270689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:15.271153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:15.273099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93ac04-6346-4a61-a5dd-afa1a8515708 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:15.298348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 674/1344] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:08:15.299073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:15.313925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b449e1d-35f7-42f0-b9a6-60cdb3cf59c4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:15.332020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cda4e-754e-4af1-9ea9-c53198a608aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:15.332020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cda4e-754e-4af1-9ea9-c53198a608aa admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.332020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cda4e-754e-4af1-9ea9-c53198a608aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.332020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cda4e-754e-4af1-9ea9-c53198a608aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.333045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cda4e-754e-4af1-9ea9-c53198a608aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:15.333447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cda4e-754e-4af1-9ea9-c53198a608aa admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:15.505747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48f713-617d-4ed7-bd6a-753e99312db3 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.506050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48f713-617d-4ed7-bd6a-753e99312db3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.506383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48f713-617d-4ed7-bd6a-753e99312db3 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.507976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 675/1345] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 17:08:15.555912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 671/1346] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:15 2022] POST /identity/v3/projects => generated 367 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:15.560592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48f713-617d-4ed7-bd6a-753e99312db3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:15.582165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.582438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.582693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.646043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:15.647885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:15.762121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:15.822452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 672/1347] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:08:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:15.823868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:15.826369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:15.881973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48f713-617d-4ed7-bd6a-753e99312db3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:15.897417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd91fd-39dd-494a-b1b0-d836a380a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:15.898340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd91fd-39dd-494a-b1b0-d836a380a022 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.898803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd91fd-39dd-494a-b1b0-d836a380a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.899253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd91fd-39dd-494a-b1b0-d836a380a022 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.900237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd91fd-39dd-494a-b1b0-d836a380a022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:15.902932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd91fd-39dd-494a-b1b0-d836a380a022 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:15.904698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045abe2-a7c3-44ff-8588-ab159b5d481c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:15.911592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:15.912588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.920355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.925048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] PATH_INFO: `/v3/users/2f0a9facc2b64d2ba4bcf899d7823b86` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:15.926874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0a9facc2b64d2ba4bcf899d7823b86)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:15.929081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:15.986378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:15.994132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 676/1348] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:15 2022] POST /identity/v3/users => generated 284 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:15.994132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:15.994132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:15.994132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:16.094128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] MFA Rules not processed for user `f6d39546d0504953a4f9d769db9421a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:16.094653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.230802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0a9facc2b64d2ba4bcf899d7823b86'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:16.234367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0a9facc2b64d2ba4bcf899d7823b86'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:16.303112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 677/1349] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:16.304798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:16.309391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.321915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0a9facc2b64d2ba4bcf899d7823b86'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:16.361270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0a9facc2b64d2ba4bcf899d7823b86'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:16.362312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0a9facc2b64d2ba4bcf899d7823b86'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:16.376060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caa0c7-6db1-4901-8516-6e8445c0ea86 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:16.393866 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:16.395905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:16.399313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:16.400336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:16.410877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 673/1350] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:15 2022] DELETE /identity/v3/users/2f0a9facc2b64d2ba4bcf899d7823b86 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:16.413286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.440826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:16.440826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.465376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:16.469155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b0c3-e1ef-4adb-b331-3a78f9945a55 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:16.520803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 678/1351] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:16.527585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3040bd-2095-4bd4-b88d-76fa437c19b2 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:16.527945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3040bd-2095-4bd4-b88d-76fa437c19b2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:16.528281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3040bd-2095-4bd4-b88d-76fa437c19b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:16.612700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ef3040bd-2095-4bd4-b88d-76fa437c19b2 None None] MFA Rules not processed for user `f6d39546d0504953a4f9d769db9421a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:16.616912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ef3040bd-2095-4bd4-b88d-76fa437c19b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.680229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771564e-7291-458c-b467-d4337ac03e51 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:16.687469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:16.689228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:16.689228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:16.689228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:16.692107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:16.694294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:16.729592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 674/1352] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:16.735293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:16.740742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.758058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668150ac-5698-4fba-a39b-313afb5c106c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:16.764576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:16.765243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:16.765528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:16.765869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] PATH_INFO: `/v3/projects/80bb962108a94de28c806813c8debb57` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:16.767142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] RBAC: Authorizing `identity:delete_project(project_id=80bb962108a94de28c806813c8debb57)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:16.769926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:16.786823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 679/1353] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:16.789333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:16.789734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:16.792697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:16.848394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] MFA Rules not processed for user `b8507de448b6455fb792bda0c949421d`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:16.849755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:16.921965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:16.934477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80bb962108a94de28c806813c8debb57', 'request_id': 'req-b864e4e3-96d2-42aa-9733-8820a6f4aede'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:16.960970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 680/1354] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:16.962268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:16.976477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee20b7-9352-43bd-b6b4-948aeda12c41 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:16.990013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:17.014066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80bb962108a94de28c806813c8debb57', 'request_id': 'req-b864e4e3-96d2-42aa-9733-8820a6f4aede'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.050101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80bb962108a94de28c806813c8debb57', 'request_id': 'req-b864e4e3-96d2-42aa-9733-8820a6f4aede'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.065208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] The token cache is being invalidate because project 80bb962108a94de28c806813c8debb57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:17.065765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.066668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.068252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 675/1355] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:16 2022] DELETE /identity/v3/projects/80bb962108a94de28c806813c8debb57 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:17.069191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:17.081826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.081923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:17.082599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.083657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.085338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.085338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:17.090654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:17.112214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 681/1356] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:17.113310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:17.115423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.130636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81f286-466a-4ede-91e1-919e8603c95f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:17.146405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:17.148109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.149177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.149470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] PATH_INFO: `/v3/projects/f10a89146d6345388ca2d5d390c472ea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.154485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10a89146d6345388ca2d5d390c472ea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:17.156342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:17.247089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864e4e3-96d2-42aa-9733-8820a6f4aede admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:17.254182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751432d-470d-494c-a524-46bde6423468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:17.270226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751432d-470d-494c-a524-46bde6423468 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.270226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751432d-470d-494c-a524-46bde6423468 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.270226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751432d-470d-494c-a524-46bde6423468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.270226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b751432d-470d-494c-a524-46bde6423468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.336300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751432d-470d-494c-a524-46bde6423468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:17.338029 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751432d-470d-494c-a524-46bde6423468 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:17.350509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 676/1357] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:17.351884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751432d-470d-494c-a524-46bde6423468 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:17.353857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b751432d-470d-494c-a524-46bde6423468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.354567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10a89146d6345388ca2d5d390c472ea', 'request_id': 'req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.364219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10a89146d6345388ca2d5d390c472ea', 'request_id': 'req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.370934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751432d-470d-494c-a524-46bde6423468 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:17.377430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10a89146d6345388ca2d5d390c472ea', 'request_id': 'req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.475350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] The token cache is being invalidate because project f10a89146d6345388ca2d5d390c472ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:17.475595 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.476695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fcdf1de8-1287-45fb-a45a-5c2fd80928ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:17.480478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 682/1358] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:17 2022] DELETE /identity/v3/projects/f10a89146d6345388ca2d5d390c472ea => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:17.484011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.485613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.487255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.529975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:17.534518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.534885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:17.535527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.536645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.536929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.538958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.552634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:17.628975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 683/1359] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:17.629966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:17.633826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.652936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb18c8-faa5-4d1d-b238-80edb0b4f85d None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:17.676381 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51806195-bbfc-4e86-af3d-9320674d1016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:17.677418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51806195-bbfc-4e86-af3d-9320674d1016 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.677883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51806195-bbfc-4e86-af3d-9320674d1016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.689307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:17.691558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51806195-bbfc-4e86-af3d-9320674d1016 admin admin] PATH_INFO: `/v3/projects/f7acd42b98fd46b4a126f9bffef16d51/users/db944ca441a44595928522929eddb91e/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.694437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:17.722507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 677/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:17.723760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:17.727955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:17.768997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51806195-bbfc-4e86-af3d-9320674d1016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7acd42b98fd46b4a126f9bffef16d51, user_id=db944ca441a44595928522929eddb91e, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:17.775258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51806195-bbfc-4e86-af3d-9320674d1016 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:17.777045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0ea77-66c4-4f48-b751-ff561d12f3fe service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:17.789716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:17.790554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.790899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.791401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] PATH_INFO: `/v3/users/12ad4c60384c4a0bb8954f69936d2510` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.797542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ad4c60384c4a0bb8954f69936d2510)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:17.799672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:17.843810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 684/1361] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:17 2022] PUT /identity/v3/projects/f7acd42b98fd46b4a126f9bffef16d51/users/db944ca441a44595928522929eddb91e/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:17.845346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3083-cf96-429c-a41c-e7fc5a273ef9 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:17.853702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3083-cf96-429c-a41c-e7fc5a273ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:17.854166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3083-cf96-429c-a41c-e7fc5a273ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:17.912406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:17.938153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-141c3083-cf96-429c-a41c-e7fc5a273ef9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:17.939406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-141c3083-cf96-429c-a41c-e7fc5a273ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.029536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ad4c60384c4a0bb8954f69936d2510'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:18.031074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ad4c60384c4a0bb8954f69936d2510'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:18.109223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ad4c60384c4a0bb8954f69936d2510'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:18.114090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 685/1362] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:18.115439 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548166f-2367-4104-b819-c0969ac78b0e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.126231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548166f-2367-4104-b819-c0969ac78b0e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.126674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548166f-2367-4104-b819-c0969ac78b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.127578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ad4c60384c4a0bb8954f69936d2510'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:18.128797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7b670a86-2084-44f0-9453-96d5f3a64067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ad4c60384c4a0bb8954f69936d2510'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:18.145901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 678/1363] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:17 2022] DELETE /identity/v3/users/12ad4c60384c4a0bb8954f69936d2510 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:18.147256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.147605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.147941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.184457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6548166f-2367-4104-b819-c0969ac78b0e None None] MFA Rules not processed for user `db944ca441a44595928522929eddb91e`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:18.186586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6548166f-2367-4104-b819-c0969ac78b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.191312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] MFA Rules not processed for user `8bb27ad53d4a454eb10268ab27ad3258`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:18.196585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.361060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 679/1364] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:18.362641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:18.367238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 686/1365] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:18.371980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c25991-0b5f-4791-ba4f-c8c2bf70cd45 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.372888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c25991-0b5f-4791-ba4f-c8c2bf70cd45 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.373908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c25991-0b5f-4791-ba4f-c8c2bf70cd45 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.382046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 687/1366] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 10 17:08:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:18.383065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c25991-0b5f-4791-ba4f-c8c2bf70cd45 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:18.385868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-79c25991-0b5f-4791-ba4f-c8c2bf70cd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.387491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:18.499374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514aa46-7cc9-4d8b-903c-40b4cc5d372a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:18.505934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:18.506339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.506679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.506960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.507626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:18.507981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:18.510157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c0cd-12c7-4186-bd4b-683d0de054fb admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:18.524175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 680/1367] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:18.526157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.526448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.526725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.529596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c25991-0b5f-4791-ba4f-c8c2bf70cd45 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:18.540609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:18.540609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.540609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.540609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.562870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:18.566119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:18.595255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 688/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:18.599457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:18.604509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.618666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:18.622961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.653631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:18.728773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 681/1369] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:08:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:18.732762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a91ad0-ad1f-44ec-b6f1-08bd1b4f7475 service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:18.735271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:18.748247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:18.752175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.752175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.752175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.752694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fedbac-6baa-4b39-bdb7-10b731a0f78d None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:18.757114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:18.758263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.758617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.758974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.759741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:18.762567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:18.779432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:18.781540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:18.817830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 689/1370] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:18.821240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:18.822115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.849430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 682/1371] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:18 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:18.849430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:18.850416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:18.855182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0888c-7a87-429a-b950-7adaa81e7987 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:18.964251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:18.965507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:18.967626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:18.967626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] PATH_INFO: `/v3/projects/3ecb448ed11f4b42a24e97cb1976ba28` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:18.973287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ecb448ed11f4b42a24e97cb1976ba28)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:18.978225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.019263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8804d-5986-43d4-a583-aa19f00404f7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.026812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:19.032852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.032852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.032852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.048758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:19.050856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.073966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 683/1372] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:19.074974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:19.104907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edb016-34f2-4f3c-a092-f2735aa8f0aa service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.111033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:19.112262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.119007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.119007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.119007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:19.119007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.142986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ecb448ed11f4b42a24e97cb1976ba28', 'request_id': 'req-1b60128a-21ab-49a6-870d-dec5c02ab4cb'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:19.154212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ecb448ed11f4b42a24e97cb1976ba28', 'request_id': 'req-1b60128a-21ab-49a6-870d-dec5c02ab4cb'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:19.178332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ecb448ed11f4b42a24e97cb1976ba28', 'request_id': 'req-1b60128a-21ab-49a6-870d-dec5c02ab4cb'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:19.197202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:19.209607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 684/1373] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:19 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:19.213876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:19.234344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0db2-c2ba-45f4-8dfe-e89526454adf admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.251722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] The token cache is being invalidate because project 3ecb448ed11f4b42a24e97cb1976ba28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:19.252954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:19.254359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b60128a-21ab-49a6-870d-dec5c02ab4cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:19.257034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 690/1374] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:18 2022] DELETE /identity/v3/projects/3ecb448ed11f4b42a24e97cb1976ba28 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:19.264506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebfb95-b7d6-4736-985d-acb640e3a92f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.264787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebfb95-b7d6-4736-985d-acb640e3a92f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.265053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebfb95-b7d6-4736-985d-acb640e3a92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.301365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8eebfb95-b7d6-4736-985d-acb640e3a92f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:19.304563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8eebfb95-b7d6-4736-985d-acb640e3a92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:19.515943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:19.515943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.515943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.515943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.518710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:19.518710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.518710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 685/1375] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:19.518710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:19.518710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:19.521116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:19.522614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699f48-9d46-4c12-870c-4b47243bad37 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.534947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:19.534947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.534947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.534947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.535958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:19.543237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.546150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 691/1376] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:19.557230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.557645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.562672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.594080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 686/1377] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:19.595984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:19.616776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778faed-8315-44f2-aebe-67526d964ea1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.628587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cef9ba-d488-44cd-98bc-6f1d35f7e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:19.629879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cef9ba-d488-44cd-98bc-6f1d35f7e59b admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.630165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cef9ba-d488-44cd-98bc-6f1d35f7e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.630441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cef9ba-d488-44cd-98bc-6f1d35f7e59b admin admin] PATH_INFO: `/v3/projects/386ecca063274422a3552966962154c9/users/0da3a55e7f3541b7abb9e54789755946/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.638254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:19.639553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:19.658997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cef9ba-d488-44cd-98bc-6f1d35f7e59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=386ecca063274422a3552966962154c9, user_id=0da3a55e7f3541b7abb9e54789755946, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:19.663435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cef9ba-d488-44cd-98bc-6f1d35f7e59b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.674808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 692/1378] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:19.678513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:19.708188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1bf71-2294-4d63-b56a-af3ca10a4936 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.724961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:19.725594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.725874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.734559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.744345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:19.744873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:19.746672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:19.772565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 687/1379] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:19 2022] PUT /identity/v3/projects/386ecca063274422a3552966962154c9/users/0da3a55e7f3541b7abb9e54789755946/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:19.777335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:19.777747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:19.778028 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:19.778430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 693/1380] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:19.783883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:19.816191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096a1d-5d4d-4b1a-9177-7b9fad814199 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:19.870089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] MFA Rules not processed for user `0da3a55e7f3541b7abb9e54789755946`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:19.876415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:20.063442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.064556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.064847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.065121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.066112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.070750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.118222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 688/1381] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:20.123049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.145849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578c3d5-afe5-4f39-a291-7c190bad59b4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.159200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.160354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.161002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.163801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.164886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.167979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.183451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 694/1382] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:19 2022] POST /identity/v3/projects => generated 379 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:20.184417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.189694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:20.356164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 689/1383] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:20 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:20.357254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.362013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5f1e4-79fa-41ee-8732-7f7336a71799 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.421826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:20.480747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.482365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37c67-b2f2-4dd3-98ed-eca2c0c57877 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.488007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.490188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.491134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.491712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.497268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.497268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.497465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.497465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.497465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.501535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:20.538731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.541762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.595141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 695/1384] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:20.596231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.609797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 690/1385] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:20 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:20.611049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.630463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e22b0-6944-427c-aaba-069b59bd7ef0 service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.644472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06ebf9-b4aa-485a-a9f7-aba1c56cc30f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.647596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.649791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.649791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.649980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.653613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.660586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.660779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.662297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.662591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.663445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.665648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.672333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.707828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 691/1386] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:20.716679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.727917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8450ee-5670-42c2-8e09-3a59471bfefa admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.736012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.741302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.741302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.741302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.741302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.741302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.758773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 692/1387] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:20.764563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.771128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eafd7-7022-4586-88f8-bf6332689714 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.777214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d503b0-6e13-401a-b9f0-fd6673c12cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.778135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d503b0-6e13-401a-b9f0-fd6673c12cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.778514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d503b0-6e13-401a-b9f0-fd6673c12cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.779087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d503b0-6e13-401a-b9f0-fd6673c12cfc admin admin] PATH_INFO: `/v3/projects/6b78463816f04792ba99a738c95ddd69/users/55134d05b9954f12af2f0e07bf109368/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.797894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 696/1388] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:20 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:20.798148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.827819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae0675-46ac-475f-ae17-eb758f40a5b1 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.827995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d503b0-6e13-401a-b9f0-fd6673c12cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b78463816f04792ba99a738c95ddd69, user_id=55134d05b9954f12af2f0e07bf109368, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.831437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d503b0-6e13-401a-b9f0-fd6673c12cfc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.836774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.837393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.837669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.840779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.841907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.844629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.864147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 697/1389] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:20.865162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:20.886398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af926-d3ed-4f9d-91f9-b1ead975986b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:20.895577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:20.895577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.895577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.895577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] PATH_INFO: `/v3/projects/582d9c1b08794b8c9cd17cd61b26daca/users/0347df9ff258477b8707308df3d66ee9/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:20.933412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582d9c1b08794b8c9cd17cd61b26daca, user_id=0347df9ff258477b8707308df3d66ee9, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:20.936498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:20.951119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:20.977869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 693/1390] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:20 2022] PUT /identity/v3/projects/6b78463816f04792ba99a738c95ddd69/users/55134d05b9954f12af2f0e07bf109368/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:20.980058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:20.980829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:20.987564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:21.037983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 698/1391] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:20 2022] PUT /identity/v3/projects/582d9c1b08794b8c9cd17cd61b26daca/users/0347df9ff258477b8707308df3d66ee9/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:21.037983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:21.038194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] MFA Rules not processed for user `55134d05b9954f12af2f0e07bf109368`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:21.039274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:21.221643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78464e-5242-42f8-ab15-611e81522008 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:21.233913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:21.235245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:21.235537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:21.235821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:21.238178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 694/1392] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:21.243387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:21.244058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:21.247453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:21.293623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549ae4e-7e9c-48d0-8ee8-32f081839cbc None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:21.299415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 699/1393] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:21.303716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:21.304340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:21.307854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:21.309634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:21.309922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:21.310200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:21.310850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:21.318829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:21.333224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efae534-6ab7-47bd-b5d0-c0da78ddd9d7 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:21.346386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:21.347373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:21.349249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:21.349708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] PATH_INFO: `/v3/users/5e09aafc4a9644d4acc2303fff71913f` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:21.354812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e09aafc4a9644d4acc2303fff71913f)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:21.357060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:21.430610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 695/1394] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:21 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:21.431566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:21.494891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:21.519655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263e926-7041-4c92-9512-1b77dda1597c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:21.525405 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:21.526837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:21.527372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:21.528224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] PATH_INFO: `/v3/projects/582d9c1b08794b8c9cd17cd61b26daca/users/0347df9ff258477b8707308df3d66ee9/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:21.530966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=582d9c1b08794b8c9cd17cd61b26daca, user_id=0347df9ff258477b8707308df3d66ee9, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:21.554256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:21.583786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 696/1395] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:21 2022] PUT /identity/v3/projects/582d9c1b08794b8c9cd17cd61b26daca/users/0347df9ff258477b8707308df3d66ee9/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:21.584481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e09aafc4a9644d4acc2303fff71913f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:21.584989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:21.694336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e09aafc4a9644d4acc2303fff71913f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:21.700417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e09aafc4a9644d4acc2303fff71913f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:21.700417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e09aafc4a9644d4acc2303fff71913f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:21.737678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e3a95615-5c05-466d-af0f-e9f46980d4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e09aafc4a9644d4acc2303fff71913f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:21.775453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4b951-7086-43da-b7a5-6efa200bf93d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:21.781327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 700/1396] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:21 2022] DELETE /identity/v3/users/5e09aafc4a9644d4acc2303fff71913f => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:21.786262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e950da8-ec36-4c73-bc50-6eab10babac8 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:21.786909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e950da8-ec36-4c73-bc50-6eab10babac8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:21.789000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e950da8-ec36-4c73-bc50-6eab10babac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:21.874022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:21.897345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3e950da8-ec36-4c73-bc50-6eab10babac8 None None] MFA Rules not processed for user `0347df9ff258477b8707308df3d66ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:21.898522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3e950da8-ec36-4c73-bc50-6eab10babac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:22.019919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.020481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.020762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.027571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.028749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.033966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.071702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 701/1397] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:22.073309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.073643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.073918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.111706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:22.117029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:22.135473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 702/1398] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:22.136175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:22.139239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:22.157783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 697/1399] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:21 2022] POST /identity/v3/users => generated 297 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:22.161527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:22.163726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:22.208607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb1d2d-ba41-4909-b74a-b055380448c1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:22.225931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.226571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.229895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.230209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.232756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.242272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.293372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 698/1400] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:22.296208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:22.303431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:22.343752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63671a7-3287-48f1-95b6-2254f8371884 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:22.359930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.361077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.361353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.361632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.367520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:22.402097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa2864-8d85-472d-8211-527d09980732 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:22.406766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.407396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.407739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.408573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.421641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.423164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.437371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 699/1401] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:22.438434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:22.444163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:22.452015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653964ed-4d1a-4848-b3da-d018c5349339 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:22.458711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.460189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.460189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.460189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] PATH_INFO: `/v3/projects/f33c28cb60214507a19f6aec845cc031/users/04a09a878e9f40fc814ccd6c5b5c57fd/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.494208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33c28cb60214507a19f6aec845cc031, user_id=04a09a878e9f40fc814ccd6c5b5c57fd, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.499084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.500578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.501706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.544219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 703/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:22.639688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 700/1403] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:22 2022] PUT /identity/v3/projects/f33c28cb60214507a19f6aec845cc031/users/04a09a878e9f40fc814ccd6c5b5c57fd/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:22.643594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:22.663945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a665ed-1701-4e1e-9aad-8ffd07ef50c2 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:22.783422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ecbd-abac-4d77-a16b-9e8bf9474ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.784103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ecbd-abac-4d77-a16b-9e8bf9474ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.785361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ecbd-abac-4d77-a16b-9e8bf9474ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.785469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ecbd-abac-4d77-a16b-9e8bf9474ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.786044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ecbd-abac-4d77-a16b-9e8bf9474ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.789309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ecbd-abac-4d77-a16b-9e8bf9474ff2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.817888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:22.829377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add66c-8de3-47b3-a2b1-afa71072d7b4 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:22.832569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 704/1404] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:22.845577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:22.846344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.846697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.847041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] PATH_INFO: `/v3/projects/f33c28cb60214507a19f6aec845cc031/users/04a09a878e9f40fc814ccd6c5b5c57fd/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.851729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33c28cb60214507a19f6aec845cc031, user_id=04a09a878e9f40fc814ccd6c5b5c57fd, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:22.854535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:22.878452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 701/1405] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:22 2022] PUT /identity/v3/projects/f33c28cb60214507a19f6aec845cc031/users/04a09a878e9f40fc814ccd6c5b5c57fd/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:22.889255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9ef7-ee46-43de-bf42-ef4355c74589 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:22.889545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9ef7-ee46-43de-bf42-ef4355c74589 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:22.889824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9ef7-ee46-43de-bf42-ef4355c74589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:22.963774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-95ca9ef7-ee46-43de-bf42-ef4355c74589 None None] MFA Rules not processed for user `04a09a878e9f40fc814ccd6c5b5c57fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:22.963774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-95ca9ef7-ee46-43de-bf42-ef4355c74589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:23.164985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:23.224656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 705/1406] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:23.298361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:23.301437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:23.418887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:23.419381 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:23.419815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:23.470542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa8b75-78a0-4c33-a461-ce749776135a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:23.471044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] MFA Rules not processed for user `b8507de448b6455fb792bda0c949421d`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:23.472607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:23.479880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:23.480551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:23.480985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:23.481425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:23.500172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:23.502339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:23.546303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 702/1407] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:23.547640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 706/1408] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:23.812317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:23.816049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:23.951838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:23.967868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5749f8e-cb77-48e1-93e1-94c273a36083 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:23.983942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:23.988844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:23.989130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:23.989400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:23.996803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:24.095022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675c290-11ff-47d9-85b9-1bb6748affde service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:24.128216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:24.129817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.130202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.130577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] PATH_INFO: `/v3/projects/f3ec355ce8444446afea97b72da044a4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.132222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3ec355ce8444446afea97b72da044a4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:24.139257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:24.229883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:24.231869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e560-d93b-4ab4-9e17-4fe17564b31e service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:24.252095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 707/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:24.364441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3ec355ce8444446afea97b72da044a4', 'request_id': 'req-5f32c9c3-e200-4c99-b924-92bea16cf62f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:24.413471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2855ad8-88a9-4cbb-ba5a-c96664840117 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.414175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2855ad8-88a9-4cbb-ba5a-c96664840117 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.414649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2855ad8-88a9-4cbb-ba5a-c96664840117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.449152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:24.466754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d2855ad8-88a9-4cbb-ba5a-c96664840117 None None] MFA Rules not processed for user `8bb27ad53d4a454eb10268ab27ad3258`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:24.470598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3ec355ce8444446afea97b72da044a4', 'request_id': 'req-5f32c9c3-e200-4c99-b924-92bea16cf62f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:24.471751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d2855ad8-88a9-4cbb-ba5a-c96664840117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:24.478336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3ec355ce8444446afea97b72da044a4', 'request_id': 'req-5f32c9c3-e200-4c99-b924-92bea16cf62f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:24.489999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:24.511884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] The token cache is being invalidate because project f3ec355ce8444446afea97b72da044a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:24.513853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:24.515212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5f32c9c3-e200-4c99-b924-92bea16cf62f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:24.520005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 703/1410] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:23 2022] DELETE /identity/v3/projects/f3ec355ce8444446afea97b72da044a4 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:24.524669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a769a5-a26a-4051-a2b8-0fb93edb8693 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.525533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a769a5-a26a-4051-a2b8-0fb93edb8693 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.526082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a769a5-a26a-4051-a2b8-0fb93edb8693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.577131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d5a769a5-a26a-4051-a2b8-0fb93edb8693 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:24.578364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d5a769a5-a26a-4051-a2b8-0fb93edb8693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:24.589632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 708/1411] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:24.664620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 704/1412] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:24.667106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.667443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.667725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.690194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d7909-778b-4ff2-8f66-392ba9b065a9 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.690745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d7909-778b-4ff2-8f66-392ba9b065a9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.691005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d7909-778b-4ff2-8f66-392ba9b065a9 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.697893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 705/1413] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 10 17:08:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:24.707890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.708214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.708487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.740156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:24.740156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:24.755971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:24.755971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:24.776339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 709/1414] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:24.780641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:24.810592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ee61f-26db-405f-912b-d4ac66788631 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:24.823158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:24.823860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.827961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.827961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.827961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:24.827961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:24.828957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:24.854208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 710/1415] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:24.855745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:24.892964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1df5d-9cf0-43ce-b477-0e9a808453b9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:24.901074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 706/1416] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:08:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:24.905465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:24.906135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.906478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.912203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:24.912415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.913341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:24.925903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:24.927651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:24.943469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1015d-6b9a-4463-94d7-568e58d0d747 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:24.958953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:24.959589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:24.959867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:24.960210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:24.987812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:24.991803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.028219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 707/1417] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:25.056635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 711/1418] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:24 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:25.060734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.075009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e495-8719-483c-b64c-abb499a6070a service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:25.199276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:25.202184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.202828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.203329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.205895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:25.208688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.318527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 708/1419] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:25 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:25.322509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.332331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.332331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:25.334698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dfbaf-5f42-4206-b078-ea493bf395a1 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:25.340123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:25.340809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.341283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.341564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.342199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:25.343805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.345977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2004-122b-4779-a451-9c7c96572cff admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:25.352154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:25.353186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.354165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.354516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] PATH_INFO: `/v3/users/b8d24868c409409fab54ca4cfcbf86ea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.355776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d24868c409409fab54ca4cfcbf86ea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:25.357123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 712/1420] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:25.360088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.361280 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.372401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06cd32-edb6-46ae-981f-f7af570a2562 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:25.379592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:25.380728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.381364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.381755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.382520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:25.384209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.398991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 713/1421] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:25.400121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.411368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4c015-7546-4f90-bc6b-d151762b212d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:25.415686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d6a5e-07b4-4dc4-951f-7caf463826c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:25.416410 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d6a5e-07b4-4dc4-951f-7caf463826c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.416690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d6a5e-07b4-4dc4-951f-7caf463826c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.416965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d6a5e-07b4-4dc4-951f-7caf463826c8 admin admin] PATH_INFO: `/v3/projects/88868d5479f547be815c388bd6bffdcd/users/a41b2e3fbca24365b62575ab9a0a663b/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.432952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d6a5e-07b4-4dc4-951f-7caf463826c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88868d5479f547be815c388bd6bffdcd, user_id=a41b2e3fbca24365b62575ab9a0a663b, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:25.435786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d6a5e-07b4-4dc4-951f-7caf463826c8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.511322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 714/1422] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:25 2022] PUT /identity/v3/projects/88868d5479f547be815c388bd6bffdcd/users/a41b2e3fbca24365b62575ab9a0a663b/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:25.522989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.523282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.523561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.554655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] MFA Rules not processed for user `a41b2e3fbca24365b62575ab9a0a663b`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:25.555948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:25.631429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d24868c409409fab54ca4cfcbf86ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:25.634460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d24868c409409fab54ca4cfcbf86ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:25.641933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 715/1423] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:25.649249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.678553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:25.741417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26cd9b-3c86-479f-8052-c10ff85f1771 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:25.746581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:25.749101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.749101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.749101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.749101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:25.750621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:25.767296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d24868c409409fab54ca4cfcbf86ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:25.783625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d24868c409409fab54ca4cfcbf86ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:25.784418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fe7b7ec9-bcf8-41c9-8385-cbea67d96c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d24868c409409fab54ca4cfcbf86ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:25.810287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 709/1424] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:25 2022] DELETE /identity/v3/users/b8d24868c409409fab54ca4cfcbf86ea => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:25.815873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:25.816371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:25.816827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:25.866993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 716/1425] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:25 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:25.871770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:25.876894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:25.889297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:25.893053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:26.014482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 710/1426] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:26.023687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:26.026843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:26.082578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247e0a-0ecf-4d2c-96c9-7ed101f46784 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:26.102063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:26.114981 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.138150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.138150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.138412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:26.140388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:26.157713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:26.203382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664caee2-c2b0-483e-8671-0031b0834b59 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:26.212222 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:26.213107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.213521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.213950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.237950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:26.242673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:26.265960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 717/1427] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:25 2022] POST /identity/v3/users => generated 297 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:26.278096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:26.278475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 711/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:26.328650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87a72d-15bd-4871-88ac-4c0de13b0840 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:26.342467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:26.343103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.345714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.345714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.345714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:26.349586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:26.391011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 718/1429] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:26.395225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:26.409487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc345c-b130-4392-9653-577e0ebf6143 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:26.460762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:26.461523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.461809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.462085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] PATH_INFO: `/v3/projects/b06ffda794e34a59aceea0916b481dd6/users/d0135b8d9f004b15800ce97118df32af/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.482460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06ffda794e34a59aceea0916b481dd6, user_id=d0135b8d9f004b15800ce97118df32af, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:26.485638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:26.543682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 712/1430] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:26 2022] PUT /identity/v3/projects/b06ffda794e34a59aceea0916b481dd6/users/d0135b8d9f004b15800ce97118df32af/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:26.544677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:26.559831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a27a9-eab1-4472-ab7f-93722fcc8cbe admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:26.684822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b473f2-2b8e-4373-8f85-a02a8d3c28d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:26.685507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b473f2-2b8e-4373-8f85-a02a8d3c28d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.685786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b473f2-2b8e-4373-8f85-a02a8d3c28d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.686555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b473f2-2b8e-4373-8f85-a02a8d3c28d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.687266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b473f2-2b8e-4373-8f85-a02a8d3c28d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:26.688954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b473f2-2b8e-4373-8f85-a02a8d3c28d3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:26.707803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 719/1431] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:26.708450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:26.742532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a6e6-e232-40e9-b184-85f5e317de08 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:26.757336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:26.758944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.758944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.759337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] PATH_INFO: `/v3/projects/b06ffda794e34a59aceea0916b481dd6/users/d0135b8d9f004b15800ce97118df32af/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.767275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06ffda794e34a59aceea0916b481dd6, user_id=d0135b8d9f004b15800ce97118df32af, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:26.770915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:26.789904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 713/1432] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:26 2022] PUT /identity/v3/projects/b06ffda794e34a59aceea0916b481dd6/users/d0135b8d9f004b15800ce97118df32af/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:26.802094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:26.802499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:26.802880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:26.850177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:26.880926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] MFA Rules not processed for user `d0135b8d9f004b15800ce97118df32af`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:26.886100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:26.911648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:26.915371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:27.053451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec1942-f6db-4273-b895-51d2f38ac630 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:27.061785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 720/1433] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:27.061901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:27.061901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:27.062185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:27.062584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] PATH_INFO: `/v3/projects/dc5beb1195b343f2b0f9ac6d20d4b0f8` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:27.064055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5beb1195b343f2b0f9ac6d20d4b0f8)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:27.066379 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:27.121933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:27.124965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:27.190815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:27.281130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416774df-f915-463a-b84a-cdb2c63d3945 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:27.286048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5beb1195b343f2b0f9ac6d20d4b0f8', 'request_id': 'req-61f8fad9-8da9-4c82-8d4a-a898d5983f51'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.361267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5beb1195b343f2b0f9ac6d20d4b0f8', 'request_id': 'req-61f8fad9-8da9-4c82-8d4a-a898d5983f51'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.370709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5beb1195b343f2b0f9ac6d20d4b0f8', 'request_id': 'req-61f8fad9-8da9-4c82-8d4a-a898d5983f51'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.394198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:27.394833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:27.395167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:27.395830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:27.397408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] The token cache is being invalidate because project dc5beb1195b343f2b0f9ac6d20d4b0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:27.400867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.402964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.406316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 714/1434] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:26 2022] DELETE /identity/v3/projects/dc5beb1195b343f2b0f9ac6d20d4b0f8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:27.410315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:27.414132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:27.526503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:27.532296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:27.548164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fad9-8da9-4c82-8d4a-a898d5983f51 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:27.552539 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 721/1435] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:27.557690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:27.558440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:27.559058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:27.559058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] PATH_INFO: `/v3/users/be12872accd94680834de30ad50db593` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:27.560580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] RBAC: Authorizing `identity:delete_user(user_id=be12872accd94680834de30ad50db593)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:27.567989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:27.796398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be12872accd94680834de30ad50db593'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.796970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be12872accd94680834de30ad50db593'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.856575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be12872accd94680834de30ad50db593'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.893036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be12872accd94680834de30ad50db593'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.896323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be12872accd94680834de30ad50db593'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:27.920185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 715/1436] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:27 2022] DELETE /identity/v3/users/be12872accd94680834de30ad50db593 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:28.253545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:28.380431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:28.387816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:28.392221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492e29c-c208-4347-aa97-5a84f6a39e5e service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:28.452297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:28.452935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.453211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.453480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] PATH_INFO: `/v3/projects/27db1ca730c74d22993885d0d5c4c353` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.454586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27db1ca730c74d22993885d0d5c4c353)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:28.456509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:28.561172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27db1ca730c74d22993885d0d5c4c353', 'request_id': 'req-a0821461-74f0-4778-9d9e-43f6602ffd80'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:28.566982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27db1ca730c74d22993885d0d5c4c353', 'request_id': 'req-a0821461-74f0-4778-9d9e-43f6602ffd80'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:28.576925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27db1ca730c74d22993885d0d5c4c353', 'request_id': 'req-a0821461-74f0-4778-9d9e-43f6602ffd80'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:28.647017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:28.648436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:28.686515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] The token cache is being invalidate because project 27db1ca730c74d22993885d0d5c4c353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:28.686805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:28.687853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a0821461-74f0-4778-9d9e-43f6602ffd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:28.689005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 722/1437] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:28 2022] DELETE /identity/v3/projects/27db1ca730c74d22993885d0d5c4c353 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:28.694120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e054-be65-4060-96e6-f40f1bc2de79 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.694403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e054-be65-4060-96e6-f40f1bc2de79 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.694674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e054-be65-4060-96e6-f40f1bc2de79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.713437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77056d-616a-4cb1-b806-f92b459426de admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:28.715224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f908e054-be65-4060-96e6-f40f1bc2de79 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:28.716447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f908e054-be65-4060-96e6-f40f1bc2de79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:28.769380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0135b8d9f004b15800ce97118df32af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06ffda794e34a59aceea0916b481dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:28.770003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.770312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.770689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.771345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:28.771652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:28.773309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:28.781312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 723/1438] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:28.789650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 716/1439] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:08:28.790264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.790552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.790842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.793533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:28.796207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:28.809543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacb202-7f54-468d-8a55-03a8a3332052 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:28.812360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:28.813573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:28.814521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f23b3d-ed32-4892-9fb9-6fb1390f938e tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0135b8d9f004b15800ce97118df32af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06ffda794e34a59aceea0916b481dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:28.815144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23b3d-ed32-4892-9fb9-6fb1390f938e tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.815454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23b3d-ed32-4892-9fb9-6fb1390f938e tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.816011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23b3d-ed32-4892-9fb9-6fb1390f938e tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.816904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23b3d-ed32-4892-9fb9-6fb1390f938e tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:28.819183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23b3d-ed32-4892-9fb9-6fb1390f938e tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:28.842447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 724/1440] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:28.842447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:28.851884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1f5f8-3479-4572-9191-7b52f6f837b4 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:28.859990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:28.860660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.860938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.861211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.861893 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:28.864446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:28.864446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:28.877803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 725/1441] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:28.890942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:28.902176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97168cd0-6431-4e15-aa8b-3a347a585d56 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:28.908863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:28.930914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.930914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.930914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.930914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:28.931273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 717/1442] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:28 2022] POST /identity/v3/projects => generated 354 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:28.931273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:28.931273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:28.932182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:28.932287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:28.932366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:28.938743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:28.939955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:28.992118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 718/1443] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Oct 10 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:28.993044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.022064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 726/1444] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:28 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:29.022907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.024696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:29.027802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:29.087425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf4667-aa88-4aea-8a36-701a452792ec None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.092462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cbcd7-d463-42dc-bba4-f8a041bdb8df admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.096511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.097119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.097397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.097670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.105870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.107269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.115975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 727/1445] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:29.116805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.172700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0135b8d9f004b15800ce97118df32af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06ffda794e34a59aceea0916b481dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.172700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.172700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.173300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.174171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:29.174576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.176631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccea28-0de7-4bbe-9a8a-1fdcb08cc772 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.196481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 719/1446] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:29.198716 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.198838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.199149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.217851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d77ca-8d8e-4643-adcb-9dc0e502316d service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.225693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.226429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.226855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.227229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.228094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.233089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.269854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:29.271089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:29.332325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 728/1447] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:29 2022] POST /identity/v3/users => generated 284 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:29.333398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.346864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8da986-729d-444c-9dcf-5dab123df715 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.350547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 720/1448] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:08:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:29.351748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.361246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0135b8d9f004b15800ce97118df32af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06ffda794e34a59aceea0916b481dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.361246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.361246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.361938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] PATH_INFO: `/v3/projects/81d5aed88e2d4100ae1f548290ebf8f2` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.361938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorizing `identity:delete_project(project_id=81d5aed88e2d4100ae1f548290ebf8f2)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.363656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.377465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aa18f-70f4-490a-8728-fc916ef12963 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.391254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.391992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.392427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.392808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.393559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.395255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.406624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 721/1449] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:29.410787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.423980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0257f-be1c-4fd0-b545-2bbb16bed119 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.430759 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.434486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.434486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.434486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.434486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.434816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.450515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 722/1450] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:29.454789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.465437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd93af2-9723-42a3-b353-f16c0e344887 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.471008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177bb30-8b9a-4630-9461-f1ce8445580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.471674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177bb30-8b9a-4630-9461-f1ce8445580a admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.475395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177bb30-8b9a-4630-9461-f1ce8445580a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.475395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177bb30-8b9a-4630-9461-f1ce8445580a admin admin] PATH_INFO: `/v3/projects/eca15c2c5c6449a695b4cee370e2e91a/users/7064f2d18dac4fdeb7604cbaa2959573/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.487681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177bb30-8b9a-4630-9461-f1ce8445580a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca15c2c5c6449a695b4cee370e2e91a, user_id=7064f2d18dac4fdeb7604cbaa2959573, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:29.490349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177bb30-8b9a-4630-9461-f1ce8445580a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:29.604263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 723/1451] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:29 2022] PUT /identity/v3/projects/eca15c2c5c6449a695b4cee370e2e91a/users/7064f2d18dac4fdeb7604cbaa2959573/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:29.614302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.615648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.615927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.622437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d5aed88e2d4100ae1f548290ebf8f2', 'request_id': 'req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:29.642797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d5aed88e2d4100ae1f548290ebf8f2', 'request_id': 'req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:29.659264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d5aed88e2d4100ae1f548290ebf8f2', 'request_id': 'req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:29.675554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] MFA Rules not processed for user `7064f2d18dac4fdeb7604cbaa2959573`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:29.675554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:29.753992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 724/1452] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:29.758473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.760482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:29.807307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:29.809936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] The token cache is being invalidate because project 81d5aed88e2d4100ae1f548290ebf8f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:29.810281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:29.811052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:29.817594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 729/1453] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:29 2022] DELETE /identity/v3/projects/81d5aed88e2d4100ae1f548290ebf8f2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:29.819088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:29.826105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:29.842527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:29.942360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3a1b4-e52e-444b-b218-73cf1a7c4b2f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.950138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:29.953984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:29.954285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:29.954568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:29.960827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:29.974183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d64d1d-587f-4a4b-a6fb-b53c91aa2cb5 tempest-VolumeQuotaClassesTest-1988764726 tempest-VolumeQuotaClassesTest-1988764726-project-admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:29.994572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:30.047670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.050576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.060953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba87ce6-0c28-41ac-be63-d9edd87ebe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.061633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba87ce6-0c28-41ac-be63-d9edd87ebe92 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.061898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba87ce6-0c28-41ac-be63-d9edd87ebe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.062338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba87ce6-0c28-41ac-be63-d9edd87ebe92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.062963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba87ce6-0c28-41ac-be63-d9edd87ebe92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.064922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba87ce6-0c28-41ac-be63-d9edd87ebe92 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.065334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 725/1454] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:08:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:08:30.195299 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 730/1455] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:29 2022] POST /identity/v3/projects => generated 369 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:30.199515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.210078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1f40-2250-44b0-b8a5-6f9cc295505c service swift] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.318792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14f90-5fe2-4e2c-a24f-c665cd0a91be None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.319084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14f90-5fe2-4e2c-a24f-c665cd0a91be None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.319344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14f90-5fe2-4e2c-a24f-c665cd0a91be None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.321079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 731/1456] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:30.328310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.329901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.330185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.334283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.340160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.340160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.340160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.340160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.340160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.358050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:30.359483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:30.421081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 726/1457] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:30 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:30.427570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.465928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88e42f-5821-4f95-b6bc-9145cd1eb55a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.488086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.488086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.488086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.488415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.489038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.491642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.506853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 732/1458] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:08:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:30.507824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.510080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:30.511899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 727/1459] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:30.515766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.530201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674fbc-de2b-49d9-bf6b-a60787d68f3b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.550485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.551263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.551551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.551829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] PATH_INFO: `/v3/projects/8b4fb13ec75b4b3f9d9065857835d87d/users/3bbd728fe95546cba2ad507ee90b8465/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.571385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4fb13ec75b4b3f9d9065857835d87d, user_id=3bbd728fe95546cba2ad507ee90b8465, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.574878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.608311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:30.638787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63ff2e-f3a9-457c-88fb-5b33c0e02586 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.643709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.644458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.644895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.645143 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.647245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:30.729218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.730649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.741108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 733/1460] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:30.742015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.743619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:30.760364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ebe3-299c-42d5-a2a6-c94bc84b0cbe service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.767991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 728/1461] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:30 2022] PUT /identity/v3/projects/8b4fb13ec75b4b3f9d9065857835d87d/users/3bbd728fe95546cba2ad507ee90b8465/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:30.772609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.885737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.886999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.887284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.887562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.893073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d5d1-c037-4931-acc9-a7668bb0baa6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.898317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.900898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.903867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9986e5-404a-4cfe-b62d-5ffcec2eebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.904589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9986e5-404a-4cfe-b62d-5ffcec2eebbb admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.904894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9986e5-404a-4cfe-b62d-5ffcec2eebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.905127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9986e5-404a-4cfe-b62d-5ffcec2eebbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.915472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9986e5-404a-4cfe-b62d-5ffcec2eebbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:30.917945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9986e5-404a-4cfe-b62d-5ffcec2eebbb admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:30.930796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 734/1462] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:30.939150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 729/1463] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:30.957443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:30.978115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a4a99-3fbe-4e20-8cb4-f96d2e4e32de service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:30.989797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:30.990490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:30.990825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:30.991108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] PATH_INFO: `/v3/projects/8b4fb13ec75b4b3f9d9065857835d87d/users/3bbd728fe95546cba2ad507ee90b8465/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:30.995035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4fb13ec75b4b3f9d9065857835d87d, user_id=3bbd728fe95546cba2ad507ee90b8465, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:31.000536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:31.028333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 735/1464] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:30 2022] PUT /identity/v3/projects/8b4fb13ec75b4b3f9d9065857835d87d/users/3bbd728fe95546cba2ad507ee90b8465/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:31.035974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:31.036553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:31.037259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:31.224875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:31.237709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] MFA Rules not processed for user `3bbd728fe95546cba2ad507ee90b8465`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:31.242221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:31.371199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 730/1465] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:31.397687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:31.432871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:31.434431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:31.492531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde466be-7c83-4325-9556-391c0182b784 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:31.496974 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50599630-700f-4e62-b548-9397e2a1ac1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:31.501193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50599630-700f-4e62-b548-9397e2a1ac1d service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:31.501193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50599630-700f-4e62-b548-9397e2a1ac1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:31.501193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50599630-700f-4e62-b548-9397e2a1ac1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:31.516055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50599630-700f-4e62-b548-9397e2a1ac1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:31.517544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50599630-700f-4e62-b548-9397e2a1ac1d service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:31.518317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f2a3-b460-4c23-8ef0-1f2d5732cb74 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:31.524499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:31.524816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:31.525536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:31.525814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:31.526084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] PATH_INFO: `/v3/users/a41b2e3fbca24365b62575ab9a0a663b` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:31.527805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41b2e3fbca24365b62575ab9a0a663b)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:31.529305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 736/1466] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:31.530057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:31.757202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41b2e3fbca24365b62575ab9a0a663b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:31.757871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41b2e3fbca24365b62575ab9a0a663b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:31.859031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41b2e3fbca24365b62575ab9a0a663b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:31.881698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41b2e3fbca24365b62575ab9a0a663b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:31.882431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41b2e3fbca24365b62575ab9a0a663b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:31.896296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 731/1467] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:31 2022] DELETE /identity/v3/users/a41b2e3fbca24365b62575ab9a0a663b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:31.905179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:31.905712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:31.905815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:31.926086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:31.927424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:32.002767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 737/1468] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:32.013069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:32.015108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:32.056867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:32.116238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb2588-3aba-4309-81dd-a5bbe502d9ae admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:32.119863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:32.120516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:32.120797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:32.121071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:32.130795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:32.132257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:32.142501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 732/1469] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:32.643659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:32.645128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:32.658235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ee3d-a8b4-49ad-8f40-c603fbc36873 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:32.662933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:32.663607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:32.663892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:32.664202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] PATH_INFO: `/v3/projects/88868d5479f547be815c388bd6bffdcd` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:32.665317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=88868d5479f547be815c388bd6bffdcd)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:32.667792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:32.807670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88868d5479f547be815c388bd6bffdcd', 'request_id': 'req-403d566c-d362-49b2-8c5a-85d05e2229ce'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:32.813313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88868d5479f547be815c388bd6bffdcd', 'request_id': 'req-403d566c-d362-49b2-8c5a-85d05e2229ce'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:32.819993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88868d5479f547be815c388bd6bffdcd', 'request_id': 'req-403d566c-d362-49b2-8c5a-85d05e2229ce'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:32.904281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] The token cache is being invalidate because project 88868d5479f547be815c388bd6bffdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:32.904578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:32.905212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:32.906745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 738/1470] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:32 2022] DELETE /identity/v3/projects/88868d5479f547be815c388bd6bffdcd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:32.910756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:32.920296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8b6a-5679-4dc9-b737-0c98741f35c6 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:32.976667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:32.977414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:32.977691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:32.977964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] PATH_INFO: `/v3/users/d0135b8d9f004b15800ce97118df32af` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:32.979091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0135b8d9f004b15800ce97118df32af)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:32.982251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:33.193592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0135b8d9f004b15800ce97118df32af'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:33.194249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0135b8d9f004b15800ce97118df32af'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:33.287403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0135b8d9f004b15800ce97118df32af'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:33.301287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0135b8d9f004b15800ce97118df32af'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:33.302010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a2e0b18-4cfd-41b5-8c6b-8b2928efe44f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0135b8d9f004b15800ce97118df32af'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:33.316883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 733/1471] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:32 2022] DELETE /identity/v3/users/d0135b8d9f004b15800ce97118df32af => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:33.827250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:33.829375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:33.845356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d566c-d362-49b2-8c5a-85d05e2229ce admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:33.952210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:33.952965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:33.953455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:33.954559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] PATH_INFO: `/v3/projects/b06ffda794e34a59aceea0916b481dd6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:33.961124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06ffda794e34a59aceea0916b481dd6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:33.961124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.004088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:34.129322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06ffda794e34a59aceea0916b481dd6', 'request_id': 'req-6f1143e7-095e-4c37-9b81-6a5a01bf124b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:34.141460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06ffda794e34a59aceea0916b481dd6', 'request_id': 'req-6f1143e7-095e-4c37-9b81-6a5a01bf124b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:34.150897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06ffda794e34a59aceea0916b481dd6', 'request_id': 'req-6f1143e7-095e-4c37-9b81-6a5a01bf124b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:34.229227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] The token cache is being invalidate because project b06ffda794e34a59aceea0916b481dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:34.229227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:34.229227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f1143e7-095e-4c37-9b81-6a5a01bf124b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:34.231103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 739/1472] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:33 2022] DELETE /identity/v3/projects/b06ffda794e34a59aceea0916b481dd6 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:34.236088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.236595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.236674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.261372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:34.264898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:34.327005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:34.366784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 734/1473] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:34.374797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.375328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.375736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.403194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:34.404476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:34.416243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 740/1474] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:34.417899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.429280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17301ad0-c33b-4726-a9d0-d5de26d53b7a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.434265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.434881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.435339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.435760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.436596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:34.437045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:34.441419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.455622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 735/1475] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:34.457364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.477316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ca702-b0eb-439c-8163-25e13193a870 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.482245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.482941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.483311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.483750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.484926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:34.487834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.581952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 741/1476] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:34 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:34.590285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.613503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc06b3d-dde1-4747-b320-e845794e1100 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.691540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.692198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.692486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.697622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.697622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:34.697622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.794703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.800219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:34.815774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33e9c-75ac-411a-b195-a650ab72d525 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.820975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.829053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.829053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.829053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] PATH_INFO: `/v3/users/7064f2d18dac4fdeb7604cbaa2959573` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.829053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7064f2d18dac4fdeb7604cbaa2959573)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:34.829053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.829481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 736/1477] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:34 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:34.830577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.841390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58983a99-1d9b-4027-bfa1-cf59e0e3abd7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.886567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.887214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.887496 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.887814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.888593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:34.901349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.925365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 737/1478] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:34.929580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.943800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cec3d2-7858-4bd1-ae42-c3f71ab6ab49 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.952269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.952269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.952269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.952269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:34.952688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:34.957632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:34.969904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 738/1479] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:34.974546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:34.988599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae17ea-7c52-4713-82b2-0514b05c5d7b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:34.995212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447f6e7-e18b-43b8-938f-744804eafbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:34.996707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447f6e7-e18b-43b8-938f-744804eafbf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:34.997219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447f6e7-e18b-43b8-938f-744804eafbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:34.997714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447f6e7-e18b-43b8-938f-744804eafbf6 admin admin] PATH_INFO: `/v3/projects/182aaec44b4945eb8b4aebec72640bcc/users/a4d4f50548cf4caa9b1f86353868d056/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.030943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447f6e7-e18b-43b8-938f-744804eafbf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182aaec44b4945eb8b4aebec72640bcc, user_id=a4d4f50548cf4caa9b1f86353868d056, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:35.033644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447f6e7-e18b-43b8-938f-744804eafbf6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:35.065917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7064f2d18dac4fdeb7604cbaa2959573'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:35.143689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 739/1480] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:34 2022] PUT /identity/v3/projects/182aaec44b4945eb8b4aebec72640bcc/users/a4d4f50548cf4caa9b1f86353868d056/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:35.155757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.156117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.156400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.167875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7064f2d18dac4fdeb7604cbaa2959573'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:35.168752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7064f2d18dac4fdeb7604cbaa2959573'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:35.169440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7064f2d18dac4fdeb7604cbaa2959573'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:35.182948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1dcc6214-d010-40ab-a1fa-089e0429c75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7064f2d18dac4fdeb7604cbaa2959573'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:35.189092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] MFA Rules not processed for user `a4d4f50548cf4caa9b1f86353868d056`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:35.190404 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:35.212782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 742/1481] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:34 2022] DELETE /identity/v3/users/7064f2d18dac4fdeb7604cbaa2959573 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:35.213145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.218247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.218247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.245616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:35.247142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:35.291662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:35.295156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 740/1482] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:35.301131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:35.305183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:35.315471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:35.350303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 743/1483] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:35.367506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:35.367506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:35.485641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114bd0-559a-4ab8-af36-a375d6944db8 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:35.489445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:35.490305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.490634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.490995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.502049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66747e0-4fe6-4999-bd50-21c53e0e20f3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:35.505228 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:35.506782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:35.509245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:35.509914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.510194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.510473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.511107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:35.514618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:35.522806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 744/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:35.635159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 741/1485] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:35 2022] POST /identity/v3/projects => generated 379 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:35.638377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:35.655042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48a13-b6dc-4dfb-bee6-50a4b8998113 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:35.759789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:35.760540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.760890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.761254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.762073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:35.764120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:35.906365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 745/1486] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:35 2022] POST /identity/v3/users => generated 296 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:35.910363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:35.924418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffc9d0-fc7f-4bdb-a3e5-4bc02937dbad admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:35.930853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:35.931817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.932318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.932773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.933798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:35.935635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:35.949333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 742/1487] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:35.953874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:35.965881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bc60b-2d83-4aa3-96df-a014d127661f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:35.972875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:35.974023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:35.974727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:35.975272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] PATH_INFO: `/v3/projects/30d9357096de403f80391ffb8ed346f8/users/59a37253267f40e58f45e3d3e09551fe/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:35.994293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] RBAC: Authorizing `identity:create_grant(project_id=30d9357096de403f80391ffb8ed346f8, user_id=59a37253267f40e58f45e3d3e09551fe, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:35.997993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:36.089366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:36.091639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:36.116770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 746/1488] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:35 2022] PUT /identity/v3/projects/30d9357096de403f80391ffb8ed346f8/users/59a37253267f40e58f45e3d3e09551fe/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:36.118457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2555b-f9c1-4a59-a70a-475697644166 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:36.120705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:36.204792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:36.205594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:36.206018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:36.206297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] PATH_INFO: `/v3/projects/eca15c2c5c6449a695b4cee370e2e91a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:36.207896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca15c2c5c6449a695b4cee370e2e91a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:36.210300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:36.249209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf8bb-0a46-42a0-be84-d501b4fa915e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:36.254488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:36.255134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:36.255415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:36.255691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:36.256559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:36.258415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:36.270093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:36.279873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 747/1489] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:36.283950 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:36.299543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c55d17-6671-413d-96be-7fdadb428ac9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:36.304777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a20429-8964-409a-a0bb-4b7c63004467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:36.305452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a20429-8964-409a-a0bb-4b7c63004467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:36.305733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a20429-8964-409a-a0bb-4b7c63004467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:36.306009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a20429-8964-409a-a0bb-4b7c63004467 admin admin] PATH_INFO: `/v3/projects/30d9357096de403f80391ffb8ed346f8/users/59a37253267f40e58f45e3d3e09551fe/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:36.307970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a20429-8964-409a-a0bb-4b7c63004467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30d9357096de403f80391ffb8ed346f8, user_id=59a37253267f40e58f45e3d3e09551fe, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:36.310703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a20429-8964-409a-a0bb-4b7c63004467 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:36.321956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 748/1490] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:36 2022] PUT /identity/v3/projects/30d9357096de403f80391ffb8ed346f8/users/59a37253267f40e58f45e3d3e09551fe/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:36.334834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:36.335169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:36.335504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:36.367793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca15c2c5c6449a695b4cee370e2e91a', 'request_id': 'req-d9b644df-1287-433a-b677-21bcd5369708'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.375039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] MFA Rules not processed for user `59a37253267f40e58f45e3d3e09551fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:36.376384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:36.452822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca15c2c5c6449a695b4cee370e2e91a', 'request_id': 'req-d9b644df-1287-433a-b677-21bcd5369708'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.460903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca15c2c5c6449a695b4cee370e2e91a', 'request_id': 'req-d9b644df-1287-433a-b677-21bcd5369708'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.475886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 749/1491] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:36.482679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:36.497647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:36.498973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] The token cache is being invalidate because project eca15c2c5c6449a695b4cee370e2e91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:36.499220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.499636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:36.499953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.504875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 743/1492] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:36 2022] DELETE /identity/v3/projects/eca15c2c5c6449a695b4cee370e2e91a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:36.505814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:36.507194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:36.579323 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6cd87-a3da-4b47-960b-3b716f307813 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:36.583152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:36.583865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:36.584262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:36.584611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:36.586679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:36.605197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:36.606612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:36.620206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b644df-1287-433a-b677-21bcd5369708 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:36.620356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 750/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:08:36.639358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:36.640089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:36.640383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:36.640648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] PATH_INFO: `/v3/users/3bbd728fe95546cba2ad507ee90b8465` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:36.641812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbd728fe95546cba2ad507ee90b8465)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:36.643793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:36.900790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbd728fe95546cba2ad507ee90b8465'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.901586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbd728fe95546cba2ad507ee90b8465'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.961679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbd728fe95546cba2ad507ee90b8465'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.975688 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbd728fe95546cba2ad507ee90b8465'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.976432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbd728fe95546cba2ad507ee90b8465'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:36.989148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 744/1494] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:36 2022] DELETE /identity/v3/users/3bbd728fe95546cba2ad507ee90b8465 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:37.225960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:37.232642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:37.250482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305b20-7c48-4ea8-9699-af95ca3a9f76 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:37.362995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:37.363664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:37.363951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:37.364262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] PATH_INFO: `/v3/users/a4d4f50548cf4caa9b1f86353868d056` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:37.365832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d4f50548cf4caa9b1f86353868d056)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:37.368231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:37.390090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:37.590045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d4f50548cf4caa9b1f86353868d056'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:37.632582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:37.634045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:37.655465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d4f50548cf4caa9b1f86353868d056'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:37.658202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d4f50548cf4caa9b1f86353868d056'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:37.667128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d4f50548cf4caa9b1f86353868d056'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:37.667281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c67e-55dc-48e8-8326-dc1fa18c08b7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:37.667281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:37.667281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:37.667546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:37.668490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] PATH_INFO: `/v3/projects/8b4fb13ec75b4b3f9d9065857835d87d` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:37.670374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4fb13ec75b4b3f9d9065857835d87d)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:37.673241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:37.696488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1d47d41-4a40-49dd-9115-d8f0371961cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d4f50548cf4caa9b1f86353868d056'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:37.723308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 751/1495] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:37 2022] DELETE /identity/v3/users/a4d4f50548cf4caa9b1f86353868d056 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:37.725857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:37.726263 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:37.726718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:37.759056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:37.760420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:37.864099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 752/1496] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:37.881948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:37.883537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:37.890441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4fb13ec75b4b3f9d9065857835d87d', 'request_id': 'req-49f20051-bc45-4213-9d54-3ec414aa1574'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:37.955938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56c0d4-7307-4ca3-b6ad-7511551c5025 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:38.005427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4fb13ec75b4b3f9d9065857835d87d', 'request_id': 'req-49f20051-bc45-4213-9d54-3ec414aa1574'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:38.018712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4fb13ec75b4b3f9d9065857835d87d', 'request_id': 'req-49f20051-bc45-4213-9d54-3ec414aa1574'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:38.046864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:38.048126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.060988 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.061283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.065380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] The token cache is being invalidate because project 8b4fb13ec75b4b3f9d9065857835d87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:38.070022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:38.070022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49f20051-bc45-4213-9d54-3ec414aa1574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:38.073130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 745/1497] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:37 2022] DELETE /identity/v3/projects/8b4fb13ec75b4b3f9d9065857835d87d => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:38.079316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ef852-232c-46a4-b770-0ab108573b1c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.079616 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ef852-232c-46a4-b770-0ab108573b1c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.079874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ef852-232c-46a4-b770-0ab108573b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.097292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:38.153647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-801ef852-232c-46a4-b770-0ab108573b1c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:38.155368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-801ef852-232c-46a4-b770-0ab108573b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:38.167765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:38.236939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 746/1498] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:38.237830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:38.239427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:38.244650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.245391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.245998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.265474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 753/1499] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:38.303652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:38.303652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:38.330190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:38.338961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 747/1500] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:38.348209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:38.355112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578fa47-13a1-4d64-ba03-4028468f5344 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:38.360277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:38.360940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.366621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.366972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.367686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:38.368005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:38.369740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:38.389929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:38.394244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 754/1501] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:38.399622 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247d336-8276-4288-8cd3-280ceb55df6f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:38.405677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:38.406200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.406465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.406769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.407417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:38.409403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:38.538211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 748/1502] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:38 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:38.542414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:38.559440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad68da-64ba-4068-9fc5-a63bb5ee3599 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:38.698311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:38.699194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.699546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.700928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.701784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:38.705075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:38.895166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 755/1503] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:38 2022] POST /identity/v3/users => generated 294 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:38.899339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:38.911434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6249d54-2f99-4ad8-b096-17f062c45533 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:38.926055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:38.926888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:38.927172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:38.927452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:38.928385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:38.930041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:38.969007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 749/1504] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:38.971930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:38.975079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:38.981466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:38.996475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef55a42-516c-4529-8f1f-a6c769e38cb8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:39.003498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c37463-aaec-43df-887a-e7193199e17d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:39.003989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:39.004856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.005333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.005653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.006508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:39.007922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:39.011014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:39.011667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.012121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.012607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] PATH_INFO: `/v3/projects/182aaec44b4945eb8b4aebec72640bcc` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.014894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182aaec44b4945eb8b4aebec72640bcc)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:39.024407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:39.027990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 756/1505] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:39.029147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:39.043187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810760-f16d-4490-8ba2-aed2465a840c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:39.049294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6e2ec-7024-4896-bc98-388739660ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:39.049992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6e2ec-7024-4896-bc98-388739660ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.050303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6e2ec-7024-4896-bc98-388739660ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.050577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6e2ec-7024-4896-bc98-388739660ab6 admin admin] PATH_INFO: `/v3/projects/d0ccee1e307345e58d8e4957fe1b38a1/users/36a1fc5fd6cb49f29b2db98f98c943e3/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.068918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6e2ec-7024-4896-bc98-388739660ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ccee1e307345e58d8e4957fe1b38a1, user_id=36a1fc5fd6cb49f29b2db98f98c943e3, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:39.071642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6e2ec-7024-4896-bc98-388739660ab6 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:39.163270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 757/1506] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:39 2022] PUT /identity/v3/projects/d0ccee1e307345e58d8e4957fe1b38a1/users/36a1fc5fd6cb49f29b2db98f98c943e3/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:39.178433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.178433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.178433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.195186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182aaec44b4945eb8b4aebec72640bcc', 'request_id': 'req-df62cde8-a21e-4cac-bd41-38ea8e462698'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.239252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] MFA Rules not processed for user `36a1fc5fd6cb49f29b2db98f98c943e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:39.239252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:39.295164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182aaec44b4945eb8b4aebec72640bcc', 'request_id': 'req-df62cde8-a21e-4cac-bd41-38ea8e462698'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.302923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182aaec44b4945eb8b4aebec72640bcc', 'request_id': 'req-df62cde8-a21e-4cac-bd41-38ea8e462698'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.316116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] The token cache is being invalidate because project 182aaec44b4945eb8b4aebec72640bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:39.316488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.317064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.318840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 750/1507] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:38 2022] DELETE /identity/v3/projects/182aaec44b4945eb8b4aebec72640bcc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:39.319717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:39.321041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:39.336268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 758/1508] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:39.341204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:39.401122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62cde8-a21e-4cac-bd41-38ea8e462698 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:39.406035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:39.406673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.406955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.407229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] PATH_INFO: `/v3/users/db944ca441a44595928522929eddb91e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.408395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=db944ca441a44595928522929eddb91e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:39.410463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:39.417385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a69bc-bcf7-468f-a8b1-e275c1e13e24 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:39.425873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:39.426793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.427794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.429291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] PATH_INFO: `/v3/users/59a37253267f40e58f45e3d3e09551fe` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.431865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a37253267f40e58f45e3d3e09551fe)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:39.435826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:39.611796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db944ca441a44595928522929eddb91e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.612400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db944ca441a44595928522929eddb91e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.651355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a37253267f40e58f45e3d3e09551fe'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.698565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db944ca441a44595928522929eddb91e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.719975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db944ca441a44595928522929eddb91e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.722503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db944ca441a44595928522929eddb91e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.737619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 751/1509] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:39 2022] DELETE /identity/v3/users/db944ca441a44595928522929eddb91e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:39.738857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:39.741002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:39.773209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a37253267f40e58f45e3d3e09551fe'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.774100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a37253267f40e58f45e3d3e09551fe'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.775748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a37253267f40e58f45e3d3e09551fe'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.794557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fa0ccba1-5864-4e6a-90f6-cc0978eaf146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a37253267f40e58f45e3d3e09551fe'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:39.809837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 759/1510] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:39 2022] DELETE /identity/v3/users/59a37253267f40e58f45e3d3e09551fe => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:39.811434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.820352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.820787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.844940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:39.846208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:39.867760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830e399-f155-458e-91e6-92ea3da52cbc admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:39.938974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:39.939669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:39.940320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:39.942134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:39.942665 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:39.959971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:39.962207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:39.996626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:40.015363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 760/1511] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:40.024064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.027572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:40.123727 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 752/1512] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:39 2022] POST /identity/v3/projects => generated 385 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:40.129682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.132891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:40.152890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8475e-9003-4919-9fc3-da965512d4bf None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.265015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28668a-ae23-444a-ab8b-d843e232386a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.279102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.279102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.279102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.279102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.279813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.279813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.279994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.280698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.283199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.285153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.296237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.297687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.327936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 761/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:40.397234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.405667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 753/1514] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:40 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:40.408192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacde52-9bcf-4f3d-a494-8aff863b60bf service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.420817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.422091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.422385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.422733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.423506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.425548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.449704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 762/1515] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:40.454101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.470502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87e96-216e-4d77-98e0-cceb91397ccb admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.478634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.479594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.479880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.480224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] PATH_INFO: `/v3/projects/197161ecc1504c5a89ff2b8a7dd17721/users/39e2132b03fa4d31ac16cae71006f424/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.507746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197161ecc1504c5a89ff2b8a7dd17721, user_id=39e2132b03fa4d31ac16cae71006f424, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.508893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.594170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.598378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 754/1516] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:40 2022] PUT /identity/v3/projects/197161ecc1504c5a89ff2b8a7dd17721/users/39e2132b03fa4d31ac16cae71006f424/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:40.599485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:40.599625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.615350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627561aa-fadf-45f7-bf0b-3e58beedadaa admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.746175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.747157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.747457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.747732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] PATH_INFO: `/v3/projects/30d9357096de403f80391ffb8ed346f8` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.749988 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30d9357096de403f80391ffb8ed346f8)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.753941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.802748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3142117-ad7a-45b9-ac5c-f9fa9b1a7272 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.815988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.817079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.817549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.818954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.820011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.822204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.845572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 755/1517] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:40.864563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:40.895728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290cc74-8ee5-4ab4-bd78-3880833f7c57 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:40.922972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0f632-4f13-4b99-a9ec-ca48d5acd555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:40.924203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0f632-4f13-4b99-a9ec-ca48d5acd555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.924669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0f632-4f13-4b99-a9ec-ca48d5acd555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.925118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0f632-4f13-4b99-a9ec-ca48d5acd555 admin admin] PATH_INFO: `/v3/projects/197161ecc1504c5a89ff2b8a7dd17721/users/39e2132b03fa4d31ac16cae71006f424/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.928537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0f632-4f13-4b99-a9ec-ca48d5acd555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197161ecc1504c5a89ff2b8a7dd17721, user_id=39e2132b03fa4d31ac16cae71006f424, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:40.932176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0f632-4f13-4b99-a9ec-ca48d5acd555 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:40.947263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 756/1518] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:40 2022] PUT /identity/v3/projects/197161ecc1504c5a89ff2b8a7dd17721/users/39e2132b03fa4d31ac16cae71006f424/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:40.958882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:40.959182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:40.959443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:40.995785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30d9357096de403f80391ffb8ed346f8', 'request_id': 'req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.009597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:41.015993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30d9357096de403f80391ffb8ed346f8', 'request_id': 'req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.027535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] MFA Rules not processed for user `39e2132b03fa4d31ac16cae71006f424`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:41.029153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.036819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30d9357096de403f80391ffb8ed346f8', 'request_id': 'req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.135117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] The token cache is being invalidate because project 30d9357096de403f80391ffb8ed346f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:41.135644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.137374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6f2ccbf1-7e06-4f64-b5b7-e5c751af46d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.141366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 763/1519] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:40 2022] DELETE /identity/v3/projects/30d9357096de403f80391ffb8ed346f8 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:41.147049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.147602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.164731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.172706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 757/1520] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:41.172706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:41.174931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.200962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:41.229695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:41.262057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.301546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4d4a2-ac02-4ebb-9a43-6b7ef3d0681d None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:41.333757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:41.333757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.333757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.333757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] PATH_INFO: `/v3/users/55134d05b9954f12af2f0e07bf109368` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.335709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55134d05b9954f12af2f0e07bf109368)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:41.335709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:41.336157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 764/1521] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:41.336157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:41.340688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.473490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4418e-3511-4147-8686-6426919dc5d7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:41.480497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:41.481815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.482757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.483475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.485961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.502045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:41.503932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87349c21-6bc2-42b8-ab6d-713a2f09dd8f service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:41.515960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 765/1522] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:41.528078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.528078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.528078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.553278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:41.554515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.576479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 766/1523] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:41.576479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:41.577865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.592747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d88373-dee1-4a7e-b103-99059c231914 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:41.597874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:41.598623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.600154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.600538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] PATH_INFO: `/v3/projects/f7acd42b98fd46b4a126f9bffef16d51` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.601804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7acd42b98fd46b4a126f9bffef16d51)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:41.603842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:41.736604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55134d05b9954f12af2f0e07bf109368'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.738052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55134d05b9954f12af2f0e07bf109368'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.745758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7acd42b98fd46b4a126f9bffef16d51', 'request_id': 'req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.751315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7acd42b98fd46b4a126f9bffef16d51', 'request_id': 'req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.757704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7acd42b98fd46b4a126f9bffef16d51', 'request_id': 'req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.795450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55134d05b9954f12af2f0e07bf109368'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.808880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55134d05b9954f12af2f0e07bf109368'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.809580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55134d05b9954f12af2f0e07bf109368'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.823031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 758/1524] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:41 2022] DELETE /identity/v3/users/55134d05b9954f12af2f0e07bf109368 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:41.823789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:41.825164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.852109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] The token cache is being invalidate because project f7acd42b98fd46b4a126f9bffef16d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:41.852407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.853062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:41.854652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 767/1525] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:41 2022] DELETE /identity/v3/projects/f7acd42b98fd46b4a126f9bffef16d51 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:41.855658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:41.857135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.896612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb009ae0-7054-460f-b1cb-0dc8dbeea6c1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:41.901526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:41.902152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.902437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.902705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] PATH_INFO: `/v3/users/36a1fc5fd6cb49f29b2db98f98c943e3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.903838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a1fc5fd6cb49f29b2db98f98c943e3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:41.905850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:41.937685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90df77-7d15-40a6-a5f2-c0ec3bb6f2cc admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:41.942648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:41.943254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.943534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.943813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.944521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:41.944827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:41.946425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c72f7-8558-464d-b560-97095ed42f21 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:41.958615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 768/1526] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:41.959930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe955a8-c660-4441-91f9-eb00fa02e200 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.960254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe955a8-c660-4441-91f9-eb00fa02e200 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.960962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe955a8-c660-4441-91f9-eb00fa02e200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:41.980138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4fe955a8-c660-4441-91f9-eb00fa02e200 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:41.984321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4fe955a8-c660-4441-91f9-eb00fa02e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:41.989489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 769/1527] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:41.990490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:41.990774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:41.991050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.011634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:42.013351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.020795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 770/1528] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:42.021556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.032568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866611b9-8ae3-45da-880f-f4312df00137 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.037226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.037825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.038132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.038407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.039034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.041609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.083597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a1fc5fd6cb49f29b2db98f98c943e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:42.084308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a1fc5fd6cb49f29b2db98f98c943e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:42.139178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 771/1529] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:42.140116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.141715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.155206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a1fc5fd6cb49f29b2db98f98c943e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:42.170167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a1fc5fd6cb49f29b2db98f98c943e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:42.171036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fd2086cb-0776-42ec-83c0-90effbce4d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a1fc5fd6cb49f29b2db98f98c943e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:42.188058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 759/1530] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:41 2022] DELETE /identity/v3/users/36a1fc5fd6cb49f29b2db98f98c943e3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:42.191813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.191813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.191813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.203690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d022a-e316-4bf9-b456-d66e668a6c90 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.211253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:42.213153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.265194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.275474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.275474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.275474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.275474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.302366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 760/1531] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:42.303221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.304563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.319473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3fede-f46a-48ab-8a65-a45dece0d3e7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.329331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.332366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.332366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.332366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.332366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.334838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.355922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.357428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec064950-36b1-47cf-868c-93cccb38af9d service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.374380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 772/1532] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:42.375527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.375812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.376115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.397777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:42.399425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.414880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 773/1533] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:42.418958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.430637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a6998-a82f-4f36-b8fb-38c0f250e243 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.436819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.437632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.438022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.438458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.439653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:42.440395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.444842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.455179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 761/1534] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:42.455179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.457820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 774/1535] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:42.458860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.462062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.472229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476fd9a-98f3-4822-b6da-d53e11056150 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.474820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02f2e0-c07f-4b2b-a56f-896065037e0d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.478994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.479349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.479676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.479991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.480136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.480289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.480386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.480590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.481478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.483013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.491651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.493136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.497341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 775/1536] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:42.497341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.505077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 762/1537] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:42.506090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.516979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4702938-e500-4c31-b796-b1f37b847728 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.517248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc74e85-b91d-4afa-bd35-a2331036ccca service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.522748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.526519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.526519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.526519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.526519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.526784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.527683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.529829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.530244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.530536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.531341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.534528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.538925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 763/1538] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:42.552415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.563876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e189-5860-4118-ac02-66453f16e4fc admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.571465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1c0fd-cd7a-4baf-95bb-4279e69f79a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.572424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1c0fd-cd7a-4baf-95bb-4279e69f79a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.572706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1c0fd-cd7a-4baf-95bb-4279e69f79a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.572982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1c0fd-cd7a-4baf-95bb-4279e69f79a7 admin admin] PATH_INFO: `/v3/projects/99ae10e564684e9c8beb47fe08afd305/users/e1127fcd66bd46ec8a98fffff4a503e6/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.593383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1c0fd-cd7a-4baf-95bb-4279e69f79a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ae10e564684e9c8beb47fe08afd305, user_id=e1127fcd66bd46ec8a98fffff4a503e6, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.597601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1c0fd-cd7a-4baf-95bb-4279e69f79a7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.664272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 776/1539] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:42.665625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.686360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:42.690358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 764/1540] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:42 2022] PUT /identity/v3/projects/99ae10e564684e9c8beb47fe08afd305/users/e1127fcd66bd46ec8a98fffff4a503e6/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:42.694422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.694773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.695048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.718154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:42.719304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:42.741515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120641b-80ce-4606-9b1a-465d330d28af admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.750068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:42.816664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.817652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.818003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.818345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.819134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.824560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.826205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 765/1541] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:42.841897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.913755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34426d-1d59-41cf-8c8e-9cbeebb5b501 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.918653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.918653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.918886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.918997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.931060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.932868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.941028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 777/1542] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:42 2022] POST /identity/v3/users => generated 288 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:42.944811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:42.948467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 766/1543] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:42.955714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e434ba3-2390-4e26-b20b-e52e7041f880 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:42.961631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:42.968290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:42.968290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:42.968290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:42.968290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:42.968290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:42.979933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 778/1544] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:42.989214 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.000865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d464fdb-cd9c-4e6c-9743-a68d2ff5237c service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.005849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.006934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.006934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.007139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.007607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.009757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.020734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 767/1545] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:43.025625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.027400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.030892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.060405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dae971-3a06-4778-9448-46bdf82269f0 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.067056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd5752-0705-49b3-b431-52daae5f5476 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.072857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.073649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.074026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.074389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] PATH_INFO: `/v3/projects/3ca52f64f7fe4e6fa89f63a8ff968751/users/56dd0b4a466f44c986babddcf9a63b43/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.076810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12089839-122e-473e-8672-f64352874d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.077645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12089839-122e-473e-8672-f64352874d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.078120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12089839-122e-473e-8672-f64352874d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.078399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12089839-122e-473e-8672-f64352874d42 admin admin] PATH_INFO: `/v3/projects/d0ccee1e307345e58d8e4957fe1b38a1` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.079787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12089839-122e-473e-8672-f64352874d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ccee1e307345e58d8e4957fe1b38a1)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.082069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12089839-122e-473e-8672-f64352874d42 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.091430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca52f64f7fe4e6fa89f63a8ff968751, user_id=56dd0b4a466f44c986babddcf9a63b43, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.094675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.109913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:43.178812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:43.188829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 779/1546] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:43 2022] PUT /identity/v3/projects/3ca52f64f7fe4e6fa89f63a8ff968751/users/56dd0b4a466f44c986babddcf9a63b43/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:43.189804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.191638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.208492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12089839-122e-473e-8672-f64352874d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ccee1e307345e58d8e4957fe1b38a1', 'request_id': 'req-12089839-122e-473e-8672-f64352874d42'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.261763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833859e6-f9f5-49f6-af86-cc591563fbcb admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.264263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12089839-122e-473e-8672-f64352874d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ccee1e307345e58d8e4957fe1b38a1', 'request_id': 'req-12089839-122e-473e-8672-f64352874d42'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.272163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12089839-122e-473e-8672-f64352874d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ccee1e307345e58d8e4957fe1b38a1', 'request_id': 'req-12089839-122e-473e-8672-f64352874d42'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.285881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12089839-122e-473e-8672-f64352874d42 admin admin] The token cache is being invalidate because project d0ccee1e307345e58d8e4957fe1b38a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:43.286182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12089839-122e-473e-8672-f64352874d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.286854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12089839-122e-473e-8672-f64352874d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.291509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 768/1547] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:43 2022] DELETE /identity/v3/projects/d0ccee1e307345e58d8e4957fe1b38a1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:43.292878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.293163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.293435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.323379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.324041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.324326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.324603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] PATH_INFO: `/v3/projects/6b78463816f04792ba99a738c95ddd69` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.325782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b78463816f04792ba99a738c95ddd69)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.327493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] MFA Rules not processed for user `56dd0b4a466f44c986babddcf9a63b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:43.327655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.328924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.399292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 769/1548] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:43.400685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.401992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.420386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b78463816f04792ba99a738c95ddd69', 'request_id': 'req-6d009414-5074-4fcf-b321-2315b35c6b84'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.420900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bee04-bf68-44cb-bb36-4ab641c6c1d8 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.425965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b78463816f04792ba99a738c95ddd69', 'request_id': 'req-6d009414-5074-4fcf-b321-2315b35c6b84'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.434796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b78463816f04792ba99a738c95ddd69', 'request_id': 'req-6d009414-5074-4fcf-b321-2315b35c6b84'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.485972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.486180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] The token cache is being invalidate because project 6b78463816f04792ba99a738c95ddd69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:43.486467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.486665 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.486958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.487236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] PATH_INFO: `/v3/users/39e2132b03fa4d31ac16cae71006f424` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.488760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e2132b03fa4d31ac16cae71006f424)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.488955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.491074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.492184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 780/1549] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:43 2022] DELETE /identity/v3/projects/6b78463816f04792ba99a738c95ddd69 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:43.493248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.496245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.582588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d009414-5074-4fcf-b321-2315b35c6b84 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.589473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.590219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.590643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.591011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.593155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.657594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.659344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.673272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 781/1550] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:43.674520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.680468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.699851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30789008-3dff-4772-a531-d8d698224fe1 service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:43.705681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e2132b03fa4d31ac16cae71006f424'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.706363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e2132b03fa4d31ac16cae71006f424'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.721510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:43.721510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:43.721510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:43.721510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] PATH_INFO: `/v3/users/04a09a878e9f40fc814ccd6c5b5c57fd` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:43.722636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a09a878e9f40fc814ccd6c5b5c57fd)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:43.724561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:43.756437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:43.807172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e2132b03fa4d31ac16cae71006f424'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.820206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e2132b03fa4d31ac16cae71006f424'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.820895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e2132b03fa4d31ac16cae71006f424'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.834650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:43.840687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 770/1551] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:43 2022] DELETE /identity/v3/users/39e2132b03fa4d31ac16cae71006f424 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:43.924444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:43.926871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:43.948649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a09a878e9f40fc814ccd6c5b5c57fd'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.995870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a09a878e9f40fc814ccd6c5b5c57fd'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.996512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a09a878e9f40fc814ccd6c5b5c57fd'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:43.997137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a09a878e9f40fc814ccd6c5b5c57fd'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.009476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25735a4a-d315-445c-a819-1502b0e450c3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:44.015431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-fba71466-b823-4673-b0ce-7f21a3f980ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a09a878e9f40fc814ccd6c5b5c57fd'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.015569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:44.016107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.016428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.016707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] PATH_INFO: `/v3/users/56dd0b4a466f44c986babddcf9a63b43` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.018066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56dd0b4a466f44c986babddcf9a63b43)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:44.021068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:44.034393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 782/1552] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:43 2022] DELETE /identity/v3/users/04a09a878e9f40fc814ccd6c5b5c57fd => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:44.246519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb94ea-0333-4a4a-9b50-cd23f013c997 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.247110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb94ea-0333-4a4a-9b50-cd23f013c997 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.247420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb94ea-0333-4a4a-9b50-cd23f013c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.254733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56dd0b4a466f44c986babddcf9a63b43'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.256264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56dd0b4a466f44c986babddcf9a63b43'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.306897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e5eb94ea-0333-4a4a-9b50-cd23f013c997 None None] MFA Rules not processed for user `e1127fcd66bd46ec8a98fffff4a503e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:44.309489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e5eb94ea-0333-4a4a-9b50-cd23f013c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.339275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56dd0b4a466f44c986babddcf9a63b43'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.365313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56dd0b4a466f44c986babddcf9a63b43'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.366145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56dd0b4a466f44c986babddcf9a63b43'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.387931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 771/1553] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:43 2022] DELETE /identity/v3/users/56dd0b4a466f44c986babddcf9a63b43 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:44.389115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:44.402575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.448267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 783/1554] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:44.452623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.452623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.452623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.476165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:44.478028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.538455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe52c7-c861-4d73-afda-6118bf64d9a5 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:44.551649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:44.552379 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.552662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.553001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] PATH_INFO: `/v3/projects/197161ecc1504c5a89ff2b8a7dd17721` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.554361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197161ecc1504c5a89ff2b8a7dd17721)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:44.558295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:44.574571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 784/1555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:44.575697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:44.577039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.594749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058352f-697a-4522-9f86-23eeaec5e944 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:44.609497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:44.611081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.613052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.613351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.615226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:44.618034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:44.730422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 785/1556] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:44 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:44.732187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:44.745064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197161ecc1504c5a89ff2b8a7dd17721', 'request_id': 'req-d2d30319-d199-4689-9a15-d475883497a3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.747286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.828083 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197161ecc1504c5a89ff2b8a7dd17721', 'request_id': 'req-d2d30319-d199-4689-9a15-d475883497a3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.829785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45c44-6d33-402d-b973-7ce0e40cc58f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:44.836075 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197161ecc1504c5a89ff2b8a7dd17721', 'request_id': 'req-d2d30319-d199-4689-9a15-d475883497a3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.836793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d1f80-cd9b-4bcf-8180-f715e0bcf6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:44.837828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d1f80-cd9b-4bcf-8180-f715e0bcf6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.838118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d1f80-cd9b-4bcf-8180-f715e0bcf6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.838393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d1f80-cd9b-4bcf-8180-f715e0bcf6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.852679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] The token cache is being invalidate because project 197161ecc1504c5a89ff2b8a7dd17721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:44.853296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.857673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:44.857673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 772/1557] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:44 2022] DELETE /identity/v3/projects/197161ecc1504c5a89ff2b8a7dd17721 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:44.857673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:44.862613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.912615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:44.946446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d1f80-cd9b-4bcf-8180-f715e0bcf6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:44.948710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d1f80-cd9b-4bcf-8180-f715e0bcf6b7 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:44.954071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:44.959094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 786/1558] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:44.960415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.960849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.961169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.983483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d30319-d199-4689-9a15-d475883497a3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:44.983886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:44.986720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:44.988632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef286b6-315c-4ede-ab26-e3fdeb3aa406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:44.989499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef286b6-315c-4ede-ab26-e3fdeb3aa406 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:44.989868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef286b6-315c-4ede-ab26-e3fdeb3aa406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:44.990145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef286b6-315c-4ede-ab26-e3fdeb3aa406 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:44.990780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef286b6-315c-4ede-ab26-e3fdeb3aa406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:44.992871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef286b6-315c-4ede-ab26-e3fdeb3aa406 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:44.996351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 787/1559] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:45.003441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.004812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.016474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507385e-72c1-43cb-bb5c-21c191956f2e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.021153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.022233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.022233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.022748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] PATH_INFO: `/v3/projects/f33c28cb60214507a19f6aec845cc031` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.023996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33c28cb60214507a19f6aec845cc031)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.026068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.125700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 773/1560] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:44 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:45.130767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.131255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.131734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.172948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:45.176191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.185485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33c28cb60214507a19f6aec845cc031', 'request_id': 'req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.211453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33c28cb60214507a19f6aec845cc031', 'request_id': 'req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.221409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33c28cb60214507a19f6aec845cc031', 'request_id': 'req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.260691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 774/1561] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:45.261487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.273300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87356af-e98a-4718-8d88-d5db9efb4e34 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.279315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.280153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.280385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.280661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.281313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.282908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.299181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 775/1562] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:45.299928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.316955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] The token cache is being invalidate because project f33c28cb60214507a19f6aec845cc031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:45.317246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.319303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.320713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed399a14-1c93-4d7f-a7c0-9f18c210ec35 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.320883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 788/1563] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:45 2022] DELETE /identity/v3/projects/f33c28cb60214507a19f6aec845cc031 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:45.325177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.327163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.328919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.330469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.330469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.330682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.331672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:45.331992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.335472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1280-051d-4e78-ad10-115d6a632c65 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.343326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02e6b-e068-48bd-88f3-3ae52c23dc38 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.351836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515ffdc-4a6d-4457-8eff-f3f9590b4c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.353491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ffdc-4a6d-4457-8eff-f3f9590b4c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.354034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ffdc-4a6d-4457-8eff-f3f9590b4c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.354390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ffdc-4a6d-4457-8eff-f3f9590b4c92 admin admin] PATH_INFO: `/v3/projects/809ba0f056184f239038878b42942cfe/users/81b13176c3e243bfb84f60a7f564cf2e/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.356817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 776/1564] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:45.358312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.358628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.358918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.379605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515ffdc-4a6d-4457-8eff-f3f9590b4c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809ba0f056184f239038878b42942cfe, user_id=81b13176c3e243bfb84f60a7f564cf2e, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.382766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515ffdc-4a6d-4457-8eff-f3f9590b4c92 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.389568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:45.390877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.404162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 777/1565] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:45.404162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.408620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.429242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd15141-1c6f-48ae-a467-c4a99b0c0db2 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.433388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.434260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.435103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.435384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.436050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.438418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.485057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 789/1566] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:45 2022] PUT /identity/v3/projects/809ba0f056184f239038878b42942cfe/users/81b13176c3e243bfb84f60a7f564cf2e/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:45.486655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.486968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.487402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.522872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:45.524390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.569068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 778/1567] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:45.569977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.630368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 790/1568] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:45.631582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.652088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512bff-3d3e-4acc-88d2-d3a1ac5882b4 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.660512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f039b0-c724-4fb5-a6c2-b8824f31d1b8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.668902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.669596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.669880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.670204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.671046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.671668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.671941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.672691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.674427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.684312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.684312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.684444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.708377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 779/1569] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:45.708377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.708377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.723364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5602e56-178d-4f7c-b94e-a397c156dc5c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.728665 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.729311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.729812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.730241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] PATH_INFO: `/v3/projects/3ca52f64f7fe4e6fa89f63a8ff968751` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.731385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca52f64f7fe4e6fa89f63a8ff968751)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.733320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.796203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 791/1570] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:45.800643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.808710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d3dfb-0a57-4064-8c24-427c4c12a02a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.813438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.814050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.814324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.814592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.815229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:45.815784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.817590 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c2df-17de-420c-a62a-e0263b9ea297 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.829829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 792/1571] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:45.830966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082a97-8560-410b-8d81-93ec46077ab6 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.838388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082a97-8560-410b-8d81-93ec46077ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.838388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082a97-8560-410b-8d81-93ec46077ab6 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.838388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 793/1572] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:08:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:45.838388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5082a97-8560-410b-8d81-93ec46077ab6 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.846173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5082a97-8560-410b-8d81-93ec46077ab6 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.850618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.851224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.851499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.851766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] PATH_INFO: `/v3/projects/809ba0f056184f239038878b42942cfe/users/81b13176c3e243bfb84f60a7f564cf2e/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.853711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=809ba0f056184f239038878b42942cfe, user_id=81b13176c3e243bfb84f60a7f564cf2e, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.856316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.860582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:45.870447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca52f64f7fe4e6fa89f63a8ff968751', 'request_id': 'req-804783f0-2243-442f-8b6f-03f0401c5bf1'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.872277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 794/1573] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:45 2022] PUT /identity/v3/projects/809ba0f056184f239038878b42942cfe/users/81b13176c3e243bfb84f60a7f564cf2e/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:45.878932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.923309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca52f64f7fe4e6fa89f63a8ff968751', 'request_id': 'req-804783f0-2243-442f-8b6f-03f0401c5bf1'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.929510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca52f64f7fe4e6fa89f63a8ff968751', 'request_id': 'req-804783f0-2243-442f-8b6f-03f0401c5bf1'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.942813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] The token cache is being invalidate because project 3ca52f64f7fe4e6fa89f63a8ff968751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:45.945850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.945850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-804783f0-2243-442f-8b6f-03f0401c5bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:45.946052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 780/1574] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:45 2022] DELETE /identity/v3/projects/3ca52f64f7fe4e6fa89f63a8ff968751 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:45.948059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc585-3c32-40b9-9741-c2df30188e46 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.948337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc585-3c32-40b9-9741-c2df30188e46 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.948610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc585-3c32-40b9-9741-c2df30188e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.954115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ed311-dc88-470a-86de-36427af280cf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:45.959653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:45.960509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:45.960795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:45.961068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:45.961803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:45.964137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:45.969786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c07fc585-3c32-40b9-9741-c2df30188e46 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:45.971040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c07fc585-3c32-40b9-9741-c2df30188e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:45.978809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 795/1575] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:45.978809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:45.980545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:46.002521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e233c3-e989-436a-93a0-79e29e1e703f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.010015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8766139-e2c5-4e3f-adf2-6bf4b6ae1198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.010706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8766139-e2c5-4e3f-adf2-6bf4b6ae1198 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.010991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8766139-e2c5-4e3f-adf2-6bf4b6ae1198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.011262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8766139-e2c5-4e3f-adf2-6bf4b6ae1198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.012125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8766139-e2c5-4e3f-adf2-6bf4b6ae1198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.014943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8766139-e2c5-4e3f-adf2-6bf4b6ae1198 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.056744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 781/1576] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:46.056744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.056744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.056744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.070608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:46.072601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:46.080766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 782/1577] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:46.081602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.082960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:46.094522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba088efe-8230-45d1-ac7e-4c83b7c4af65 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.099267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.099866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.100186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.100462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.101100 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.102838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.112766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 783/1578] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:46.113618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.115211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:46.130993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdd623-1d5a-4169-bf68-ef961662dda6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.138068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 796/1579] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:45 2022] POST /identity/v3/projects => generated 357 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:46.138068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.138068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.138068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.138439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.138439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.138439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.138680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.148466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.149593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.161818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 784/1580] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:46.162438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.163262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:46.164623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:46.172844 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1437260-ad9c-44cf-b7e7-3bdde1396a86 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.248719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 797/1581] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:46.248719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.249704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.250587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.250970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.252840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] PATH_INFO: `/v3/projects/9e67dedca0c848b193b097d8c797469b/users/d702ba76d9054e40905b01cc805caf3c/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.274862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e67dedca0c848b193b097d8c797469b, user_id=d702ba76d9054e40905b01cc805caf3c, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.278907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.287276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cab8f-8a2b-4354-bac7-8704ca2d5ba1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.301573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee40703-698e-48d1-af73-b0e8df38ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.302572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee40703-698e-48d1-af73-b0e8df38ee17 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.302572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee40703-698e-48d1-af73-b0e8df38ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.305202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee40703-698e-48d1-af73-b0e8df38ee17 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.305202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee40703-698e-48d1-af73-b0e8df38ee17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.305734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee40703-698e-48d1-af73-b0e8df38ee17 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.350155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 785/1582] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:46 2022] PUT /identity/v3/projects/9e67dedca0c848b193b097d8c797469b/users/d702ba76d9054e40905b01cc805caf3c/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:46.351001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.363570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:46.371217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 798/1583] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:46 2022] POST /identity/v3/users => generated 279 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:46.372815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.373104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.373376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.405383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] MFA Rules not processed for user `d702ba76d9054e40905b01cc805caf3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:46.406618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:46.439265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c48a9-a2e4-4db9-af04-75dc00e86f85 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.445532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.446509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.446917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.447295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.448592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:46.449311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.453138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.470331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 786/1584] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:46.471577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.487839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158251f-5670-4c3a-88fd-aba8a215dbef admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.493493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.494772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.495059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.495334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.495725 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 799/1585] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:46.496205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.497007 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.497946 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.508584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c09fcf-4399-4ac2-9ee4-46d0d2aa7394 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.516521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 787/1586] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:46.516690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.516690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.516690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.516690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.516992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.519283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:46.525614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:46.549157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d453894-6b75-4db9-8c0b-1aa72fcf9377 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:46.554715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:46.555456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:46.555741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:46.556765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:46.557485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:46.560484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:47.225063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 800/1587] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:46 2022] POST /identity/v3/projects => generated 373 bytes in 729 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:47.229557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:47.332640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf12be1-e980-46e7-841f-4c92794da6e3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:47.332640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:47.332640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:47.332640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:47.333008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:47.333008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:47.333008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:47.333008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 801/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:47.333008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:47.337309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4b739-2246-4b91-b167-41d45e42444c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:47.353695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:47.354345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:47.354627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:47.354905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:47.355556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:47.357516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:47.721000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 788/1589] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:46 2022] POST /identity/v3/projects => generated 387 bytes in 1196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:47.721674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:47.746459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 802/1590] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:47 2022] POST /identity/v3/users => generated 287 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:47.746459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:47.760979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a54539-8d7d-4179-a7e5-5249706d68a3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:47.814615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480288ad-177c-4006-a5fa-18c2df5e4ae3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:47.822859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:47.823874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:47.824277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:47.824807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] PATH_INFO: `/v3/projects/4e01433c0cd24e698630188f5d55dae6/users/d8e4b83624ff40408e9904827b7614d7/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:47.875547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e01433c0cd24e698630188f5d55dae6, user_id=d8e4b83624ff40408e9904827b7614d7, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:47.879786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:47.898859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca678ef-aac4-4bae-a3de-dbba3a2013ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:47.898859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca678ef-aac4-4bae-a3de-dbba3a2013ac admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:47.899420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca678ef-aac4-4bae-a3de-dbba3a2013ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:47.900417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca678ef-aac4-4bae-a3de-dbba3a2013ac admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:47.901301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca678ef-aac4-4bae-a3de-dbba3a2013ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:47.903576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca678ef-aac4-4bae-a3de-dbba3a2013ac admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:47.976218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 789/1591] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:47 2022] PUT /identity/v3/projects/4e01433c0cd24e698630188f5d55dae6/users/d8e4b83624ff40408e9904827b7614d7/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:47.976218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.010121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 803/1592] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:47 2022] POST /identity/v3/users => generated 300 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:48.011370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702844f0-5b33-4630-92df-141cbb40895e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.011728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702844f0-5b33-4630-92df-141cbb40895e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.015663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702844f0-5b33-4630-92df-141cbb40895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.057098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-702844f0-5b33-4630-92df-141cbb40895e None None] MFA Rules not processed for user `d8e4b83624ff40408e9904827b7614d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:48.058712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-702844f0-5b33-4630-92df-141cbb40895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:48.140315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed7205-185e-44f6-bc05-40c96a8bb819 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.154364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.155900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.159908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.159908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.159908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.162276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.186844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 804/1593] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:48.186844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702844f0-5b33-4630-92df-141cbb40895e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.198829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702844f0-5b33-4630-92df-141cbb40895e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.201483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 790/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:48.202441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.205334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.206377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.207497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.208052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.208801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.210485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.228225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb449e-86df-4396-975a-41fae7b099ba admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.233893 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 805/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:48.235888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.237941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.241233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.241233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.241233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.242366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.244877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.256122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:48.279324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50afc3-a8e5-4af8-92f5-2c537c329bf4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.297011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.297719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.298808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.298808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.299092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.304019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.324501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 806/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:48.325875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.347083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a768141-b0b7-4fb1-8d98-3fdf88d23d2b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.352289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.353192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.353566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.353947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] PATH_INFO: `/v3/projects/2271c12badb54d28b8f6782ae85ba6c3/users/b61511caf65142b1a0f8724513e4a6eb/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.374293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 791/1597] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/projects => generated 355 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:48.374293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.399473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2271c12badb54d28b8f6782ae85ba6c3, user_id=b61511caf65142b1a0f8724513e4a6eb, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.405082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.470547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61151e-81c0-4839-9da7-62a922ebe1e2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.477001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.477998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.481027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.481027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] PATH_INFO: `/v3/projects/741b2e3c05224086b569304d6ae3682a/users/0c32c28566da4cccaf6c0b123fd9fc2e/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.505549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 807/1598] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:48 2022] PUT /identity/v3/projects/2271c12badb54d28b8f6782ae85ba6c3/users/b61511caf65142b1a0f8724513e4a6eb/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:48.510128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.512169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=741b2e3c05224086b569304d6ae3682a, user_id=0c32c28566da4cccaf6c0b123fd9fc2e, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.516979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.583901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 792/1599] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:48 2022] PUT /identity/v3/projects/741b2e3c05224086b569304d6ae3682a/users/0c32c28566da4cccaf6c0b123fd9fc2e/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:48.587763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.599612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e518-6268-427d-8181-e23a0efe2f74 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.669503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971efec-feec-4d7c-9a2f-fc4efa830b3b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.674717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36c164-4061-46b4-8a05-09152eaac831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.675452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36c164-4061-46b4-8a05-09152eaac831 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.675645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36c164-4061-46b4-8a05-09152eaac831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.675926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36c164-4061-46b4-8a05-09152eaac831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.676602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36c164-4061-46b4-8a05-09152eaac831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.678205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36c164-4061-46b4-8a05-09152eaac831 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.678304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.678835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.679139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.679537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.680220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.683098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.691500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 793/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:48.693145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.693436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.693712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.729195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] MFA Rules not processed for user `0c32c28566da4cccaf6c0b123fd9fc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:48.730357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:48.807614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 794/1601] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:48.808523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.820772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 808/1602] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/users => generated 284 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:48.821784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.824146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320757ab-e915-407b-b466-f3f858b38b7f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.829206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.829883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.830160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.830432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] PATH_INFO: `/v3/projects/2271c12badb54d28b8f6782ae85ba6c3/users/b61511caf65142b1a0f8724513e4a6eb/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.832485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2271c12badb54d28b8f6782ae85ba6c3, user_id=b61511caf65142b1a0f8724513e4a6eb, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.835571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.835779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e8226-4b77-4088-8d7b-1f02595fd7da admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.841701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe702ab4-5aa0-4cd4-9da2-d36ba666a7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.844359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe702ab4-5aa0-4cd4-9da2-d36ba666a7c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.844359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe702ab4-5aa0-4cd4-9da2-d36ba666a7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.844359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe702ab4-5aa0-4cd4-9da2-d36ba666a7c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.844359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe702ab4-5aa0-4cd4-9da2-d36ba666a7c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.846767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe702ab4-5aa0-4cd4-9da2-d36ba666a7c0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.847994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 795/1603] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:48 2022] PUT /identity/v3/projects/2271c12badb54d28b8f6782ae85ba6c3/users/b61511caf65142b1a0f8724513e4a6eb/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:48.848993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:48.919179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:48.943096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbf4a9-51cf-4b4f-b0c2-14dcdfd50859 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:48.948063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ed0c9-cb31-4822-9d1d-d0282436cb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:48.948708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ed0c9-cb31-4822-9d1d-d0282436cb39 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.948989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ed0c9-cb31-4822-9d1d-d0282436cb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.949260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ed0c9-cb31-4822-9d1d-d0282436cb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.949901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ed0c9-cb31-4822-9d1d-d0282436cb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:48.951494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ed0c9-cb31-4822-9d1d-d0282436cb39 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:48.963827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 796/1604] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:48.964618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 809/1605] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:48.965857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.966140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.966491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:48.966785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:48.966954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:48.967048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.001361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] MFA Rules not processed for user `81b13176c3e243bfb84f60a7f564cf2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:49.002621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:49.005098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] MFA Rules not processed for user `b61511caf65142b1a0f8724513e4a6eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:49.006383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:49.084601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 810/1606] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:49.085442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.097390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 797/1607] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:49.098286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.152274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4caf-c262-451a-b214-9ecb5b717aed None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.156951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.157568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.157849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.158132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.158766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.160707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.160852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4273-12b1-409f-a19d-86f47deb6cfc None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.166245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.166879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.167161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.167433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] PATH_INFO: `/v3/projects/f8952e8a7cc94f2ca3ab880eeb5f83c6/users/4622da02634444749e8d1010b64f18fc/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.181550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8952e8a7cc94f2ca3ab880eeb5f83c6, user_id=4622da02634444749e8d1010b64f18fc, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.184306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.275731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 811/1608] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:49 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:49.276687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.278346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:49.310136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 798/1609] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:49 2022] PUT /identity/v3/projects/f8952e8a7cc94f2ca3ab880eeb5f83c6/users/4622da02634444749e8d1010b64f18fc/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:49.310982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.312561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:49.335782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b915f-23f3-4903-bbf4-74879dc73c11 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.369160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b1b5e-a53d-4d7c-9fb1-f8079747860e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.373041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.373760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.374048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.374321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.388817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.389440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.389722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.389998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.391825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:49.436358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.437792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.448566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 799/1610] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:49.449545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.455283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.456736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.470313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:49.473813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 812/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:49.474497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.530666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b2cba-0680-41af-b6d4-32b30e5a0499 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.536861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.537756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.538057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.538330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.539858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.541904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.560585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 800/1612] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:49.562868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c1c7d-238b-4384-93d9-4a9085f5a66d service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.564515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.577666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.578574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.578855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.579112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.579865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.584468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.589839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5f77-be5c-4cec-875e-00dfb6ac8d18 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.598064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 813/1613] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:49.598962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.603661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3460a-1a05-4943-b92b-cc5066e37435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.604376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3460a-1a05-4943-b92b-cc5066e37435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.604661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3460a-1a05-4943-b92b-cc5066e37435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.604929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3460a-1a05-4943-b92b-cc5066e37435 admin admin] PATH_INFO: `/v3/projects/06939da2aa95417b9c954b72202263d2/users/e2628deba0f84b77b3f0bb2bf3634f03/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.610766 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624362d-2484-4a5f-b6ed-ca0f99f5b393 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.616014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.616714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.617200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.617509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] PATH_INFO: `/v3/projects/f8952e8a7cc94f2ca3ab880eeb5f83c6/users/4622da02634444749e8d1010b64f18fc/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.622230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8952e8a7cc94f2ca3ab880eeb5f83c6, user_id=4622da02634444749e8d1010b64f18fc, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.622845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.634758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3460a-1a05-4943-b92b-cc5066e37435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06939da2aa95417b9c954b72202263d2, user_id=e2628deba0f84b77b3f0bb2bf3634f03, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.637518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3460a-1a05-4943-b92b-cc5066e37435 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.640599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 814/1614] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:49 2022] PUT /identity/v3/projects/f8952e8a7cc94f2ca3ab880eeb5f83c6/users/4622da02634444749e8d1010b64f18fc/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:49.643801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.717567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95558f1-4283-44da-ad2d-c9cc9e69c243 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.723636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.723636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.723636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.723636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] PATH_INFO: `/v3/users/d702ba76d9054e40905b01cc805caf3c` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.724520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d702ba76d9054e40905b01cc805caf3c)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.726430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.751991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 801/1615] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:49 2022] PUT /identity/v3/projects/06939da2aa95417b9c954b72202263d2/users/e2628deba0f84b77b3f0bb2bf3634f03/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:49.754117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.754480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.754822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.785134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] MFA Rules not processed for user `4622da02634444749e8d1010b64f18fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:49.786412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:49.871649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:49.892658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 802/1616] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:49.894172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.917838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd702ba76d9054e40905b01cc805caf3c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:49.961799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701448cd-efc9-4448-abe8-43fe8404b238 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:49.967258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:49.968161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:49.968432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:49.968882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:49.969636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:49.971695 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:49.986851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 803/1617] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:49.986851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:49.989727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd702ba76d9054e40905b01cc805caf3c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:49.990522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd702ba76d9054e40905b01cc805caf3c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:49.991208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd702ba76d9054e40905b01cc805caf3c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:50.006380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd702ba76d9054e40905b01cc805caf3c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:50.021630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 815/1618] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:49 2022] DELETE /identity/v3/users/d702ba76d9054e40905b01cc805caf3c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:50.022594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:50.024046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.049308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec690-7d5c-4bd7-a6f2-d4fbc477e304 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.099678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad035430-1225-45d3-91c2-bad0739b94aa admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.105020 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff2fd3-9515-4a32-855b-52e392b16086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.105786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff2fd3-9515-4a32-855b-52e392b16086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.105890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.106172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff2fd3-9515-4a32-855b-52e392b16086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.106502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.106598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff2fd3-9515-4a32-855b-52e392b16086 admin admin] PATH_INFO: `/v3/projects/06939da2aa95417b9c954b72202263d2/users/e2628deba0f84b77b3f0bb2bf3634f03/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.106780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.107048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.108889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.109191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff2fd3-9515-4a32-855b-52e392b16086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06939da2aa95417b9c954b72202263d2, user_id=e2628deba0f84b77b3f0bb2bf3634f03, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.112162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff2fd3-9515-4a32-855b-52e392b16086 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.140194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 816/1619] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:50 2022] PUT /identity/v3/projects/06939da2aa95417b9c954b72202263d2/users/e2628deba0f84b77b3f0bb2bf3634f03/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:50.150332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:50.152341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.152863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.153365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.184726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:50.185991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.205539 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.207112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7f14-37b5-4ec6-bc00-d6318904b103 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.223738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 804/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:50.225038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.225323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.225662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.259597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 817/1621] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:50.269713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:50.269900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] MFA Rules not processed for user `e2628deba0f84b77b3f0bb2bf3634f03`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:50.271070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.271231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.331041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cc787-b231-45a8-b43b-addbb2f0f41c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.335400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.336119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.336743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.337136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.345456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 805/1622] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:50.346877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.348419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.357591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:50.359052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.364138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 818/1623] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:50.417779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e153f-872a-4ede-a7a8-3873ed1b52bb None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.421487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.422067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.422346 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.422642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.433629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.434612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.442497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:50.444241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.446986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 806/1624] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:08:50.460782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca196526-16a8-43c5-94e6-6cbfd08a05da service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.465011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.465904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.466265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.466689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.468580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.527522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.529008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.537969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 819/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:08:50.658663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:50.700872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:50.709229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b041960-e329-4e7b-832d-06e19267563f service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.713376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.718532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.718532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.718532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.718532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.768403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:50.774336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:50.776495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:50.780055 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.781532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.791696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 807/1626] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:50.808414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb9fe1-c6f5-41eb-b914-15adaf9048a6 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:50.808414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:50.808414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:50.808414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:50.809066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] PATH_INFO: `/v3/projects/9e67dedca0c848b193b097d8c797469b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:50.809066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e67dedca0c848b193b097d8c797469b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:50.810660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:50.971194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e67dedca0c848b193b097d8c797469b', 'request_id': 'req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:50.977374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e67dedca0c848b193b097d8c797469b', 'request_id': 'req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:50.984059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e67dedca0c848b193b097d8c797469b', 'request_id': 'req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.097523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] The token cache is being invalidate because project 9e67dedca0c848b193b097d8c797469b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:51.097819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.098439 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.099959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 820/1627] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:50 2022] DELETE /identity/v3/projects/9e67dedca0c848b193b097d8c797469b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:51.103964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:51.114659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca7505-6f6e-4ae3-8da6-03a2deb6b096 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:51.173081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:51.173731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:51.175518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:51.175859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] PATH_INFO: `/v3/users/b61511caf65142b1a0f8724513e4a6eb` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:51.177005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61511caf65142b1a0f8724513e4a6eb)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:51.180921 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:51.252693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:51.281320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:51.356666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61511caf65142b1a0f8724513e4a6eb'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.357659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61511caf65142b1a0f8724513e4a6eb'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.497740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61511caf65142b1a0f8724513e4a6eb'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.511746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61511caf65142b1a0f8724513e4a6eb'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.512474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0746617f-f79d-4104-81e2-8cacd7b477f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61511caf65142b1a0f8724513e4a6eb'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:51.526687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 808/1628] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:08:51 2022] DELETE /identity/v3/users/b61511caf65142b1a0f8724513e4a6eb => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:51.780337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:51.904475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:52.014749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:52.017643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.038746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a1ba0-067d-435f-9c97-1f6541e8c1bf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:52.106203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:52.107564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.108055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.111965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] PATH_INFO: `/v3/projects/2271c12badb54d28b8f6782ae85ba6c3` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.111965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2271c12badb54d28b8f6782ae85ba6c3)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:52.112573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:52.246825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117538c-ef6e-41c5-bea7-a167772b31e9 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.247112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117538c-ef6e-41c5-bea7-a167772b31e9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.247397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117538c-ef6e-41c5-bea7-a167772b31e9 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.248796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 809/1629] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:08:52.260295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf326704-139b-4cde-8f78-7067c52d8406 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.260580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf326704-139b-4cde-8f78-7067c52d8406 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.260852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf326704-139b-4cde-8f78-7067c52d8406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.267492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:52.282427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2271c12badb54d28b8f6782ae85ba6c3', 'request_id': 'req-e74e1433-7eb4-41b4-8a26-c59929f10ad8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:52.283210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bf326704-139b-4cde-8f78-7067c52d8406 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:52.284601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bf326704-139b-4cde-8f78-7067c52d8406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.291741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2271c12badb54d28b8f6782ae85ba6c3', 'request_id': 'req-e74e1433-7eb4-41b4-8a26-c59929f10ad8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:52.300118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2271c12badb54d28b8f6782ae85ba6c3', 'request_id': 'req-e74e1433-7eb4-41b4-8a26-c59929f10ad8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:52.323593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:52.374725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 810/1630] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:08:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:52.400657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] The token cache is being invalidate because project 2271c12badb54d28b8f6782ae85ba6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:08:52.400958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:52.403788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:08:52.406811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 821/1631] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:08:52 2022] DELETE /identity/v3/projects/2271c12badb54d28b8f6782ae85ba6c3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:52.419847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f059-99d2-411f-bb97-c3b7987bb338 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.423525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f059-99d2-411f-bb97-c3b7987bb338 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.423884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f059-99d2-411f-bb97-c3b7987bb338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.451907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:52.453467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.469725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a7e3f059-99d2-411f-bb97-c3b7987bb338 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:52.473881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a7e3f059-99d2-411f-bb97-c3b7987bb338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.483017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e1433-7eb4-41b4-8a26-c59929f10ad8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:52.509748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:52.509748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.509748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.509748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.514353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.562608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:52.564936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:52.582137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 822/1632] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:52.583061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:52.584579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.598207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 811/1633] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:52.607391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.607679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.607956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.657269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:52.658807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.669419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 812/1634] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:08:52.673614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:52.683626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf1801-76d8-41a4-95c5-d6dde79fcd9f service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:52.685960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a37f0-f6a4-4d06-9507-cc3072b851a3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:52.687548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:52.688196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.688477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.688748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.690463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:52.692690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:52.693544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.694024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.694507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.695467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:08:52.695972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:52.697841 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:52.714487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 813/1635] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:08:52.722368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:52.738856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831b1b9-c205-45b6-a13f-42aefe609642 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:52.746184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:52.747627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.747911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.748219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.748908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:52.751840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:52.788804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:52.791802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:52.804549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:52.820085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 823/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:08:52.894623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 814/1637] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:52 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:52.899108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:52.911009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:52.916629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8ab7f-93ac-4efa-ab38-40c85b0d22b2 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:52.978334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:52.978951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:52.980101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:52.980101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:52.980325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:52.982111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.060067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 824/1638] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:52 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:53.063460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.074709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0735f0e-2475-4a39-a8e7-8746f08b2b69 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.079484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.080279 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.080561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.080832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.081541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.083147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.097924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 815/1639] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:53.102616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.113536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50909a56-dea4-4101-9465-9d078c33db7a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.118264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd886c-7b0a-4144-98a0-4efa8ba05507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.118945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd886c-7b0a-4144-98a0-4efa8ba05507 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.119374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd886c-7b0a-4144-98a0-4efa8ba05507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.119677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd886c-7b0a-4144-98a0-4efa8ba05507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.120454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd886c-7b0a-4144-98a0-4efa8ba05507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.122144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd886c-7b0a-4144-98a0-4efa8ba05507 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.132459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 825/1640] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:53.138188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.157213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb18df-06ab-480a-afee-c57fa56bc768 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.162092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.162880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.163223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.163569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] PATH_INFO: `/v3/projects/5e3f908b9ce940efafb95df1696ec5a3/users/299dd23270224afc8f2957cb5147be56/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.179169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3f908b9ce940efafb95df1696ec5a3, user_id=299dd23270224afc8f2957cb5147be56, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.182072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.285295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 816/1641] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:53 2022] PUT /identity/v3/projects/5e3f908b9ce940efafb95df1696ec5a3/users/299dd23270224afc8f2957cb5147be56/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:53.302604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.308838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.308838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.348787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] MFA Rules not processed for user `299dd23270224afc8f2957cb5147be56`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:53.350023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:53.430017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 826/1642] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:53.435515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.494943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:53.567677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8c138-b150-4740-8c8b-8295c431af18 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.586595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.587954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.589074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.592120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.593439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.596837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:53.597287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.698445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 817/1643] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:53 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:53.702088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.720748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416aff-ed42-4a1a-91b2-bd7d22244174 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.793229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.793898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.794230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.794509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.795137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.797069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.923026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 827/1644] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:53 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:53.926700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.937882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1e989-4c5b-45a5-98a7-45dc57361059 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.945221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.946031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.946500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.946912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.947758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.949603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:53.961649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 818/1645] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:08:53.965743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:53.976970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b3cc2-4d8a-4669-a237-6a8a98053cbf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:53.982947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1b13d-cf69-4e8d-b5b2-f71e37c47b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:53.983555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b13d-cf69-4e8d-b5b2-f71e37c47b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:53.984196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b13d-cf69-4e8d-b5b2-f71e37c47b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:53.984196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b13d-cf69-4e8d-b5b2-f71e37c47b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:53.984774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1b13d-cf69-4e8d-b5b2-f71e37c47b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:53.987349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1b13d-cf69-4e8d-b5b2-f71e37c47b3d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.003962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 828/1646] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:54.005627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.017352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1246b-d6c5-4063-b587-e0abdf30c0de admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.022304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.023655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.023971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.024717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] PATH_INFO: `/v3/projects/033b40f361574c03b2c6686eed62ac31/users/44459e9c5a88417e91bf8269f6ae9012/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.041501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033b40f361574c03b2c6686eed62ac31, user_id=44459e9c5a88417e91bf8269f6ae9012, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.044308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.134648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 819/1647] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:54 2022] PUT /identity/v3/projects/033b40f361574c03b2c6686eed62ac31/users/44459e9c5a88417e91bf8269f6ae9012/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:54.145207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.145733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.145733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.175020 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] MFA Rules not processed for user `44459e9c5a88417e91bf8269f6ae9012`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:54.176279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:54.244522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 829/1648] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:54.249155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.313151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014bf03-eec6-4d60-8506-74f0f947bd62 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.318008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.318609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.318889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.319165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.319796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.321747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.399256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 820/1649] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:08:54 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:08:54.400746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.411188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b7d31-28d3-4e2b-bde5-1d988ebf00d7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.474888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.475512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.475794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.476105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.476760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.479466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.575364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 830/1650] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:08:54 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:08:54.579190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.610571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa599f-4ba3-4a64-b85d-c1d6c6b1ef1c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.610571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.610571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.610571 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.610884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.610884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.610884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.626274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 821/1651] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:54.627079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.638594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67b2c-f6a3-4e1b-ba9a-f47082d3cbc5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.646602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.646602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.646602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.646602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] PATH_INFO: `/v3/projects/7de6efc57d134e2f88986a500781167b/users/74e998970c1e49e2a7026c81535fdf57/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.677257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de6efc57d134e2f88986a500781167b, user_id=74e998970c1e49e2a7026c81535fdf57, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.680199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.762920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 831/1652] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:54 2022] PUT /identity/v3/projects/7de6efc57d134e2f88986a500781167b/users/74e998970c1e49e2a7026c81535fdf57/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:08:54.767081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.786316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683899-be4c-4510-8f8b-026958d4b975 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.788680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:54.862273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979504f-798c-45d2-921f-7b4d6787f449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.863058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979504f-798c-45d2-921f-7b4d6787f449 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.863497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979504f-798c-45d2-921f-7b4d6787f449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.863782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979504f-798c-45d2-921f-7b4d6787f449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.864479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979504f-798c-45d2-921f-7b4d6787f449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.868228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979504f-798c-45d2-921f-7b4d6787f449 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.880067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 822/1653] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:08:54.884986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:54.895699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262142-f547-4db5-ac09-369520033fc5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:54.902153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:54.902842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.903129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.903421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] PATH_INFO: `/v3/projects/7de6efc57d134e2f88986a500781167b/users/74e998970c1e49e2a7026c81535fdf57/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.905586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de6efc57d134e2f88986a500781167b, user_id=74e998970c1e49e2a7026c81535fdf57, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:54.908394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:54.920300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 832/1654] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:08:54 2022] PUT /identity/v3/projects/7de6efc57d134e2f88986a500781167b/users/74e998970c1e49e2a7026c81535fdf57/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:08:54.931458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:54.932016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:54.932238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:54.963936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] MFA Rules not processed for user `74e998970c1e49e2a7026c81535fdf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:08:54.965158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:54.995940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:55.037871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 823/1655] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:08:55.131492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:08:55.133418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:08:55.197247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f844fc1-5394-4c98-a36c-a428fb9df6df admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:08:55.200870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:08:55.201475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:08:55.201750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:08:55.202018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:08:55.390487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:08:55.392216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:08:55.403954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 833/1656] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:08:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:08:55.469744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:56.264520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:56.378586 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:56.777840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:56.806086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:08:56.955213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:02.888767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:02.898322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d162-ebec-479c-9baf-2062d0b8f596 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:02.901955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:02.902563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:02.902840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:02.903120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:02.951465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:02.960096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:02.961469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:02.970267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 824/1657] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:05.289457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:05.300558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d5b38-17d0-48af-8d50-b9c2f37e75d4 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:05.304243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec0e1d-b64c-4ad8-b358-834a48c2258d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:05.304839 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec0e1d-b64c-4ad8-b358-834a48c2258d service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:05.305115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec0e1d-b64c-4ad8-b358-834a48c2258d service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:05.305384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec0e1d-b64c-4ad8-b358-834a48c2258d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:05.314458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec0e1d-b64c-4ad8-b358-834a48c2258d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:05.316165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec0e1d-b64c-4ad8-b358-834a48c2258d service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:05.326554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 834/1658] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:09.331314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:09.331786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:09.415009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac79-b0e5-4b02-bf2e-f3adf12b7c2e service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:09.421362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:09.422081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:09.422570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:09.422768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] PATH_INFO: `/v3/users/0da3a55e7f3541b7abb9e54789755946` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:09.424271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da3a55e7f3541b7abb9e54789755946)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:09.426285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:09.672435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da3a55e7f3541b7abb9e54789755946'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:09.684421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da3a55e7f3541b7abb9e54789755946'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:09.755389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da3a55e7f3541b7abb9e54789755946'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:09.769501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da3a55e7f3541b7abb9e54789755946'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:09.770087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da3a55e7f3541b7abb9e54789755946'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:09.785368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 825/1659] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:09 2022] DELETE /identity/v3/users/0da3a55e7f3541b7abb9e54789755946 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:09.792223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:09.792223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:09.792223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:09.811718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:09.813125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:09.881412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 835/1660] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:09.892715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:09.894215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:09.956978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5019a-28f9-44d7-92f3-ad397ae581ff admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:09.960742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:09.962193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:09.962479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:09.962749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:09.979896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:09.980990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:09.993892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 826/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:10.482382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:10.482382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:10.493095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1315e-ac6a-44e5-a056-df2e77f5be99 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:10.499054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:10.499871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:10.500065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:10.500341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] PATH_INFO: `/v3/projects/386ecca063274422a3552966962154c9` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:10.501550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] RBAC: Authorizing `identity:delete_project(project_id=386ecca063274422a3552966962154c9)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:10.503517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:10.613608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:10.616504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386ecca063274422a3552966962154c9', 'request_id': 'req-b72bd726-5251-4d0c-9698-d55fe93ef83d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:10.622566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386ecca063274422a3552966962154c9', 'request_id': 'req-b72bd726-5251-4d0c-9698-d55fe93ef83d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:10.629471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386ecca063274422a3552966962154c9', 'request_id': 'req-b72bd726-5251-4d0c-9698-d55fe93ef83d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:10.709376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] The token cache is being invalidate because project 386ecca063274422a3552966962154c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:10.709703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:10.710345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:10.715711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 836/1662] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:10 2022] DELETE /identity/v3/projects/386ecca063274422a3552966962154c9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:10.717049 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:10.743260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2b7c7-2f98-4adc-99da-7a3eb2584753 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:10.743657 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:10.792570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:10.793523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:10.793819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:10.794096 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] PATH_INFO: `/v3/users/0347df9ff258477b8707308df3d66ee9` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:10.795230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0347df9ff258477b8707308df3d66ee9)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:10.797189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:11.006313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0347df9ff258477b8707308df3d66ee9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.006967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0347df9ff258477b8707308df3d66ee9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.059671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:11.061256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:11.074272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd726-5251-4d0c-9698-d55fe93ef83d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:11.075971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0347df9ff258477b8707308df3d66ee9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.078946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:11.079577 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.079853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.080179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] PATH_INFO: `/v3/users/299dd23270224afc8f2957cb5147be56` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.081329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299dd23270224afc8f2957cb5147be56)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:11.085717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:11.092296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0347df9ff258477b8707308df3d66ee9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.093060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-71549344-a27d-4289-b94b-0167cbcd4564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0347df9ff258477b8707308df3d66ee9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.111909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 827/1663] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:10 2022] DELETE /identity/v3/users/0347df9ff258477b8707308df3d66ee9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:11.319596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299dd23270224afc8f2957cb5147be56'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.384216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299dd23270224afc8f2957cb5147be56'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.385615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299dd23270224afc8f2957cb5147be56'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.386389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299dd23270224afc8f2957cb5147be56'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.399186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299dd23270224afc8f2957cb5147be56'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.412273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 837/1664] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:11 2022] DELETE /identity/v3/users/299dd23270224afc8f2957cb5147be56 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:11.416875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.417226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.417621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.437871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:11.439071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:11.507103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 828/1665] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:11.517469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:11.519013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:11.520993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:11.522554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:11.546580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea9bff-338a-4793-a1c8-a26b7d5954c5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:11.552176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:11.552812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.553086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.553353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] PATH_INFO: `/v3/projects/582d9c1b08794b8c9cd17cd61b26daca` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.554490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=582d9c1b08794b8c9cd17cd61b26daca)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:11.556392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:11.560998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:11.588161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188fde8-2d9d-4bcd-91ae-a78828dcd33e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:11.591742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f895-4417-43f2-b6c6-b49a27d0a21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:11.592399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f895-4417-43f2-b6c6-b49a27d0a21f service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.592756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f895-4417-43f2-b6c6-b49a27d0a21f service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.593010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f895-4417-43f2-b6c6-b49a27d0a21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.602147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f895-4417-43f2-b6c6-b49a27d0a21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:11.603594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f895-4417-43f2-b6c6-b49a27d0a21f service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:11.612861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 829/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:11.694779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582d9c1b08794b8c9cd17cd61b26daca', 'request_id': 'req-35f384e6-33ad-4524-8ff6-403eadbcdf5e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.701106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582d9c1b08794b8c9cd17cd61b26daca', 'request_id': 'req-35f384e6-33ad-4524-8ff6-403eadbcdf5e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.707433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582d9c1b08794b8c9cd17cd61b26daca', 'request_id': 'req-35f384e6-33ad-4524-8ff6-403eadbcdf5e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.760498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] The token cache is being invalidate because project 582d9c1b08794b8c9cd17cd61b26daca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:11.760732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.761374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-35f384e6-33ad-4524-8ff6-403eadbcdf5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:11.762912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 838/1667] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:11 2022] DELETE /identity/v3/projects/582d9c1b08794b8c9cd17cd61b26daca => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:11.767808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.768309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.768386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.789610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:11.790808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:11.889525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 830/1668] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:11.897565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.897868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.898188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.919601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:11.920909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:11.928928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 839/1669] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:11.933913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:11.944171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6671e7-ebf8-45da-b47a-e101acb08738 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:11.948937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:11.949752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.950033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.950941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.951628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:11.951931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:11.953559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:11.966358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 831/1670] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:11.971154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:11.981652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a884a-b585-4c0a-a411-8fe41bbc3b11 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:11.986491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:11.987086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:11.987423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:11.987675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:11.988382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:11.990240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.012396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.014252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:12.027351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f2f8-24d2-4fbd-987b-8360efacdf2e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.036944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.037737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.038133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.038525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] PATH_INFO: `/v3/projects/5e3f908b9ce940efafb95df1696ec5a3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.039711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3f908b9ce940efafb95df1696ec5a3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.042231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.098333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=21,keystone:SELECT=75,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:12.105217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 840/1671] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:11 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:12.109238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.169592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703ad89-e7fe-43cf-8ee9-ba616660fa9f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.174223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.174827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.175110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.175386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.176078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.177969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.201784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3f908b9ce940efafb95df1696ec5a3', 'request_id': 'req-023f0aaa-4c5e-424d-95eb-ab32d50f3252'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.269708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3f908b9ce940efafb95df1696ec5a3', 'request_id': 'req-023f0aaa-4c5e-424d-95eb-ab32d50f3252'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.288492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3f908b9ce940efafb95df1696ec5a3', 'request_id': 'req-023f0aaa-4c5e-424d-95eb-ab32d50f3252'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.288682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 841/1672] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:12 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:12.288682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.302494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287e877-7ccb-4a89-a5f4-9ae3a75f2783 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.307767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] The token cache is being invalidate because project 5e3f908b9ce940efafb95df1696ec5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:12.321450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.321450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.321450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 832/1673] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:12 2022] DELETE /identity/v3/projects/5e3f908b9ce940efafb95df1696ec5a3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:12.325012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.327726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:12.378644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.379256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.379535 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.379810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.380477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.382065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.387401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:12.392431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 842/1674] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:12.396525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.403422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:12.409121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f0aaa-4c5e-424d-95eb-ab32d50f3252 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.415741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085209e2-c9ae-4050-a1d1-5a881c02f796 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.415856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.416217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.416589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.416869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] PATH_INFO: `/v3/users/44459e9c5a88417e91bf8269f6ae9012` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.418350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44459e9c5a88417e91bf8269f6ae9012)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.420453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.421065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.421744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.425499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.425499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.425499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.425499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.440175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 843/1675] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:12.445505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.457037 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aef6b-ed96-49a3-94da-03b3cfd67185 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.461831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3417ba-557f-4495-940d-ffacdab59705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.462505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3417ba-557f-4495-940d-ffacdab59705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.462802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3417ba-557f-4495-940d-ffacdab59705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.464059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3417ba-557f-4495-940d-ffacdab59705 admin admin] PATH_INFO: `/v3/projects/e55e9e8c787e4d189efdfea564852298/users/45a7d19c205445ccaa60cd38c3c1631b/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.477895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3417ba-557f-4495-940d-ffacdab59705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55e9e8c787e4d189efdfea564852298, user_id=45a7d19c205445ccaa60cd38c3c1631b, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.480647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3417ba-557f-4495-940d-ffacdab59705 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.554954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 844/1676] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:12 2022] PUT /identity/v3/projects/e55e9e8c787e4d189efdfea564852298/users/45a7d19c205445ccaa60cd38c3c1631b/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:12.566146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.566433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.566720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.594730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] MFA Rules not processed for user `45a7d19c205445ccaa60cd38c3c1631b`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:12.595974 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:12.616401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44459e9c5a88417e91bf8269f6ae9012'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.617107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44459e9c5a88417e91bf8269f6ae9012'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.664411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 845/1677] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:12.669433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.670712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:12.734513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd02f9-6851-4f07-8324-0602999857a9 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:12.739345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:12.739978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:12.740283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:12.740552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:12.741176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:12.743074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:12.755883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:12.783718 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44459e9c5a88417e91bf8269f6ae9012'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.797142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44459e9c5a88417e91bf8269f6ae9012'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.797848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44459e9c5a88417e91bf8269f6ae9012'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:12.810506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 833/1678] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:12 2022] DELETE /identity/v3/users/44459e9c5a88417e91bf8269f6ae9012 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:12.882992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 846/1679] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:12 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:12.887173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:12.954529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8390cde-30fc-4186-9c3e-69bb25bee3a7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.015224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.015842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.016150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.016424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.017058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.018986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:13.449129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:13.449646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 834/1680] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:12 2022] POST /identity/v3/users => generated 297 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:13.453575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:13.453734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:13.466696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f5234-4b5a-4639-8693-5f6059a1742f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.471847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.473049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.473560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.474065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] PATH_INFO: `/v3/projects/033b40f361574c03b2c6686eed62ac31` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.476165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033b40f361574c03b2c6686eed62ac31)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.480487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:13.486647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81cb71-baf6-4e78-a23e-e6bd88560b81 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.497299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.498643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.499024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.499395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.500186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.503886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:13.520095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 835/1681] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:13.522093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:13.532499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c29dde-8d4e-404e-b429-8aac0f8e98a8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.537220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.552962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.552962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.552962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] PATH_INFO: `/v3/projects/c031bf3c14104617939b3578b508594b/users/23ba6e5bbda54e75aef8e7d3071ead1b/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.555133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c031bf3c14104617939b3578b508594b, user_id=23ba6e5bbda54e75aef8e7d3071ead1b, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.558466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:13.652677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:13.666268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 836/1682] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:13 2022] PUT /identity/v3/projects/c031bf3c14104617939b3578b508594b/users/23ba6e5bbda54e75aef8e7d3071ead1b/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:13.671506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:13.688134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033b40f361574c03b2c6686eed62ac31', 'request_id': 'req-578bab7d-96ae-4abf-9c81-7d33e8113e02'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:13.694660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033b40f361574c03b2c6686eed62ac31', 'request_id': 'req-578bab7d-96ae-4abf-9c81-7d33e8113e02'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:13.702081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033b40f361574c03b2c6686eed62ac31', 'request_id': 'req-578bab7d-96ae-4abf-9c81-7d33e8113e02'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:13.761256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab14eb-fb3e-4dd4-a4f2-b85b7e2d24da admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.789402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] The token cache is being invalidate because project 033b40f361574c03b2c6686eed62ac31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:13.789402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:13.789402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:13.789887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 847/1683] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:13 2022] DELETE /identity/v3/projects/033b40f361574c03b2c6686eed62ac31 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:13.797005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:13.798609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:13.816292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.816936 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.817192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.817465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.818160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.819749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:13.832103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 837/1684] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:13.835236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:13.835487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:13.848235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3d479-ba1a-488e-bb34-bde00c1bbd32 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.853234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c862-63a5-4808-8882-5ae6c06a0672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.853893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c862-63a5-4808-8882-5ae6c06a0672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.854022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:13.854173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c862-63a5-4808-8882-5ae6c06a0672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.855240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c862-63a5-4808-8882-5ae6c06a0672 admin admin] PATH_INFO: `/v3/projects/c031bf3c14104617939b3578b508594b/users/23ba6e5bbda54e75aef8e7d3071ead1b/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.857346 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c862-63a5-4808-8882-5ae6c06a0672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c031bf3c14104617939b3578b508594b, user_id=23ba6e5bbda54e75aef8e7d3071ead1b, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.861600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c862-63a5-4808-8882-5ae6c06a0672 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:13.875995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 838/1685] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:13 2022] PUT /identity/v3/projects/c031bf3c14104617939b3578b508594b/users/23ba6e5bbda54e75aef8e7d3071ead1b/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:13.882806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bab7d-96ae-4abf-9c81-7d33e8113e02 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:13.887306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.887965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.888915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.920552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] MFA Rules not processed for user `23ba6e5bbda54e75aef8e7d3071ead1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:13.921819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:13.950801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:13.951443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:13.951771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:13.952132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] PATH_INFO: `/v3/users/74e998970c1e49e2a7026c81535fdf57` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:13.953282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e998970c1e49e2a7026c81535fdf57)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:13.955248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:14.009276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 839/1686] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:14.021801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:14.023179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:14.081334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbe637-46cd-4a98-bb07-f0040984d265 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:14.085420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:14.086046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:14.086332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:14.086603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:14.097142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:14.098600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:14.108130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 840/1687] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:14.167917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e998970c1e49e2a7026c81535fdf57'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.212618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:14.221951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb6a35-fefd-471a-89e9-d74bbb0e0fbb service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:14.225772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:14.226369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:14.226651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:14.226926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:14.228729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:14.240470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e998970c1e49e2a7026c81535fdf57'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.241089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e998970c1e49e2a7026c81535fdf57'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.241699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e998970c1e49e2a7026c81535fdf57'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.256066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e998970c1e49e2a7026c81535fdf57'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.271265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 848/1688] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:13 2022] DELETE /identity/v3/users/74e998970c1e49e2a7026c81535fdf57 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:14.293474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:14.295935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72419682-cbf4-4dc8-b37d-f0e9b7127a05 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:14.306478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 841/1689] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:14.412258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:14.701009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:14.702572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:14.780331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8902b-8b3d-4678-8ae4-0079233e87cf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:14.787444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:14.788120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:14.788400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:14.788671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] PATH_INFO: `/v3/projects/7de6efc57d134e2f88986a500781167b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:14.789943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de6efc57d134e2f88986a500781167b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:14.791824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:14.811062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:14.908199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de6efc57d134e2f88986a500781167b', 'request_id': 'req-2b2ba155-7d1a-49b0-9945-d0261925e7f4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.913918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de6efc57d134e2f88986a500781167b', 'request_id': 'req-2b2ba155-7d1a-49b0-9945-d0261925e7f4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:14.920781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de6efc57d134e2f88986a500781167b', 'request_id': 'req-2b2ba155-7d1a-49b0-9945-d0261925e7f4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:15.051081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] The token cache is being invalidate because project 7de6efc57d134e2f88986a500781167b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:15.051081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:15.051081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2b2ba155-7d1a-49b0-9945-d0261925e7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:15.056735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 849/1690] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:14 2022] DELETE /identity/v3/projects/7de6efc57d134e2f88986a500781167b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:15.059102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.059912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.060370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.094825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:15.096052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:15.201581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 842/1691] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:15.209048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.209337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.209618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.237781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:15.240183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:15.253239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 850/1692] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:15.254666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.269276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a060b-77a5-477c-a589-bf4631002f00 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:15.275508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:15.280651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.280651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.280651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.280651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:15.280651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:15.280651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:15.296148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 843/1693] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:15.299461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.308882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fd6e8-9a29-4661-9ecd-261975a212f4 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:15.314510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:15.315271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.315548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.316707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.317430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:15.319559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:15.431617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 851/1694] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:15 2022] POST /identity/v3/projects => generated 399 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:15.435777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.448104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd897f-d4b6-4069-a90c-150db0f57780 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:15.456269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:15.512358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:15.512837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.513162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.513431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.514067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:15.516000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:15.603292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 844/1695] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:15 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:15.607490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.618055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d2fbd-97ee-4024-a65d-c193ad1481c4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:15.623202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:15.623874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.624180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.624470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.625109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:15.626706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:15.637919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 852/1696] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:15.641757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.660152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe9553-a194-4324-96cf-801bd4b0179b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:15.660152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec9cca-32fa-47d3-ae6a-e36f2b582fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:15.660152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec9cca-32fa-47d3-ae6a-e36f2b582fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.660152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec9cca-32fa-47d3-ae6a-e36f2b582fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.660519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec9cca-32fa-47d3-ae6a-e36f2b582fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.660519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec9cca-32fa-47d3-ae6a-e36f2b582fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:15.661866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec9cca-32fa-47d3-ae6a-e36f2b582fb5 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:15.675080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 845/1697] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:15.678167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.688077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd3522-54c9-4bb7-b747-bf7fbfc49def admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:15.693244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:15.694012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.694209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.694442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] PATH_INFO: `/v3/projects/a9e6089ccc5246b4ba26b2282c1c3ac1/users/ef45c86c5a8e46379b116ab095d2f0ef/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.711174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e6089ccc5246b4ba26b2282c1c3ac1, user_id=ef45c86c5a8e46379b116ab095d2f0ef, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:15.715202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:15.824230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 853/1698] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:15 2022] PUT /identity/v3/projects/a9e6089ccc5246b4ba26b2282c1c3ac1/users/ef45c86c5a8e46379b116ab095d2f0ef/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:15.834869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:15.835157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:15.835435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:15.867635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] MFA Rules not processed for user `ef45c86c5a8e46379b116ab095d2f0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:15.868669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:15.933830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 846/1699] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:15.977612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:15.979579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:16.040597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba400-04b1-41f8-90cf-f54c641e87ea admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:16.044290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:16.045213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:16.045498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:16.045765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:16.054057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:16.055513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:16.064743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 854/1700] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:19.076652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:19.078344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:19.147113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff16fb-f8f3-4465-bdec-e2b91e4975b6 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:19.152216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:19.152839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:19.153115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:19.153429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] PATH_INFO: `/v3/users/e1127fcd66bd46ec8a98fffff4a503e6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:19.154527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1127fcd66bd46ec8a98fffff4a503e6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:19.156444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:19.179705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:19.375605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1127fcd66bd46ec8a98fffff4a503e6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.376464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1127fcd66bd46ec8a98fffff4a503e6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.452194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1127fcd66bd46ec8a98fffff4a503e6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.466681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:19.468505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:19.485169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1127fcd66bd46ec8a98fffff4a503e6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.486065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1468c78f-5ab5-4941-8871-0e0c5ed58005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1127fcd66bd46ec8a98fffff4a503e6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.486215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64715ef8-8e4d-4b7e-a5f0-3d585a36b85e service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:19.491907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:19.492518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:19.492796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:19.493067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] PATH_INFO: `/v3/users/45a7d19c205445ccaa60cd38c3c1631b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:19.494353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a7d19c205445ccaa60cd38c3c1631b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:19.496718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:19.502005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 847/1701] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:19 2022] DELETE /identity/v3/users/e1127fcd66bd46ec8a98fffff4a503e6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:19.666499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:19.672830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a7d19c205445ccaa60cd38c3c1631b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.771849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a7d19c205445ccaa60cd38c3c1631b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.773559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a7d19c205445ccaa60cd38c3c1631b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.773559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a7d19c205445ccaa60cd38c3c1631b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.793260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a7d19c205445ccaa60cd38c3c1631b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:19.825849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 855/1702] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:19 2022] DELETE /identity/v3/users/45a7d19c205445ccaa60cd38c3c1631b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:19.830433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:19.830892 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:19.831266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:19.852257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:19.853594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:19.920474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 848/1703] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:19.930239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:19.931657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:19.994532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465403a-100e-425f-9f51-e6db354c1b06 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:19.998648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:19.999248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:19.999522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:19.999793 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:20.006882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:20.008344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:20.011682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:20.013295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:20.021573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9529ccb-d68c-487a-8776-aa2eb5bca1a7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:20.026414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 856/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:20.026639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:20.027881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:20.027881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:20.027881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] PATH_INFO: `/v3/projects/99ae10e564684e9c8beb47fe08afd305` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:20.028984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ae10e564684e9c8beb47fe08afd305)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:20.030898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:20.139286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ae10e564684e9c8beb47fe08afd305', 'request_id': 'req-d9113152-0f55-446f-bdbb-4eff1728a38c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.206048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ae10e564684e9c8beb47fe08afd305', 'request_id': 'req-d9113152-0f55-446f-bdbb-4eff1728a38c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.212477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ae10e564684e9c8beb47fe08afd305', 'request_id': 'req-d9113152-0f55-446f-bdbb-4eff1728a38c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.225693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] The token cache is being invalidate because project 99ae10e564684e9c8beb47fe08afd305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:20.226007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.226662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.228272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 849/1705] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:20 2022] DELETE /identity/v3/projects/99ae10e564684e9c8beb47fe08afd305 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:20.232469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:20.248665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031c1f8-9f89-4695-908d-9baf513fd27c service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:20.311959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:20.312632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:20.312909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:20.313643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] PATH_INFO: `/v3/users/81b13176c3e243bfb84f60a7f564cf2e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:20.314439 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b13176c3e243bfb84f60a7f564cf2e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:20.316734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:20.431839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:20.434257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:20.458671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9113152-0f55-446f-bdbb-4eff1728a38c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:20.465120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:20.466105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:20.466451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:20.466786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] PATH_INFO: `/v3/projects/e55e9e8c787e4d189efdfea564852298` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:20.468062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55e9e8c787e4d189efdfea564852298)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:20.470201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:20.494641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:20.508387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b13176c3e243bfb84f60a7f564cf2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.567112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b13176c3e243bfb84f60a7f564cf2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.567783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b13176c3e243bfb84f60a7f564cf2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.568568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b13176c3e243bfb84f60a7f564cf2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.576256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:20.585127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b13176c3e243bfb84f60a7f564cf2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.602402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 857/1706] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:20 2022] DELETE /identity/v3/users/81b13176c3e243bfb84f60a7f564cf2e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:20.603496 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55e9e8c787e4d189efdfea564852298', 'request_id': 'req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.675775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55e9e8c787e4d189efdfea564852298', 'request_id': 'req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.688001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55e9e8c787e4d189efdfea564852298', 'request_id': 'req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.704696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] The token cache is being invalidate because project e55e9e8c787e4d189efdfea564852298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:20.705350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.706119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:20.708742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 850/1707] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:20 2022] DELETE /identity/v3/projects/e55e9e8c787e4d189efdfea564852298 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:20.712708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:20.714461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:20.791326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d2644-fad1-4c48-a5d2-4069f3816c1c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:20.796213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:20.796962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:20.797288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:20.797633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] PATH_INFO: `/v3/users/23ba6e5bbda54e75aef8e7d3071ead1b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:20.798784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ba6e5bbda54e75aef8e7d3071ead1b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:20.800734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:20.908588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:20.908588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.028016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c7d4e-a470-4dbc-ac06-74550a71a9a1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.037760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ba6e5bbda54e75aef8e7d3071ead1b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.037874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.037874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.037874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.037874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] PATH_INFO: `/v3/projects/809ba0f056184f239038878b42942cfe` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.038210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=809ba0f056184f239038878b42942cfe)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.039643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.103296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ba6e5bbda54e75aef8e7d3071ead1b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.104010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ba6e5bbda54e75aef8e7d3071ead1b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.106513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ba6e5bbda54e75aef8e7d3071ead1b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.117565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ba6e5bbda54e75aef8e7d3071ead1b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.130899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 858/1708] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:20 2022] DELETE /identity/v3/users/23ba6e5bbda54e75aef8e7d3071ead1b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:21.131666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.134845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.145254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809ba0f056184f239038878b42942cfe', 'request_id': 'req-cbccb6bc-795e-499f-9500-1562fd1e4614'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.199560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:21.203515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07813ca4-cfa4-4a04-85be-1c6264431618 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.217424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809ba0f056184f239038878b42942cfe', 'request_id': 'req-cbccb6bc-795e-499f-9500-1562fd1e4614'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.225124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809ba0f056184f239038878b42942cfe', 'request_id': 'req-cbccb6bc-795e-499f-9500-1562fd1e4614'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.238209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] The token cache is being invalidate because project 809ba0f056184f239038878b42942cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:21.238583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.239354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cbccb6bc-795e-499f-9500-1562fd1e4614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.240989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 851/1709] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:20 2022] DELETE /identity/v3/projects/809ba0f056184f239038878b42942cfe => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:21.245703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a324d-d83c-4723-b802-80d59b39b2e1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.252492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a324d-d83c-4723-b802-80d59b39b2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.252492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a324d-d83c-4723-b802-80d59b39b2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.261180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.261807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.262082 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.262352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] PATH_INFO: `/v3/users/d8e4b83624ff40408e9904827b7614d7` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.263479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e4b83624ff40408e9904827b7614d7)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.265387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.265530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3d3a324d-d83c-4723-b802-80d59b39b2e1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:21.267046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3d3a324d-d83c-4723-b802-80d59b39b2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.275121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 852/1710] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:21.283256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.283631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.283916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.305062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:21.306391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.314999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 853/1711] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:21.319101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.328264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122352b-26ee-4fe0-ba5e-202d50ca1eda None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.332987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.333592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.336575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.336575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.336575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:21.336575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.337101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.351619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 854/1712] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:09:21.356674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.366452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a0dd2-881f-461d-ba91-a0feb0501cdf admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.371306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.371909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.372358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.372618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.373255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.375223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.381224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:21.485595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 855/1713] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:21.486928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.489136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e4b83624ff40408e9904827b7614d7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.546357 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bb696-56f1-41e4-99a2-165ce24d44f2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.551551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf0e7f-5b05-4880-8fe6-210591ea3b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.552195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf0e7f-5b05-4880-8fe6-210591ea3b36 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.552475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf0e7f-5b05-4880-8fe6-210591ea3b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.552743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf0e7f-5b05-4880-8fe6-210591ea3b36 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.553564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf0e7f-5b05-4880-8fe6-210591ea3b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.555456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf0e7f-5b05-4880-8fe6-210591ea3b36 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.567053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e4b83624ff40408e9904827b7614d7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.567941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e4b83624ff40408e9904827b7614d7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.568998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e4b83624ff40408e9904827b7614d7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.582754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e4b83624ff40408e9904827b7614d7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.595324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 859/1714] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:21 2022] DELETE /identity/v3/users/d8e4b83624ff40408e9904827b7614d7 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:21.596998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.598455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.645888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 856/1715] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:21.645888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.645888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.645888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.668645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:21.669838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.673040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05140d0-7516-49f0-9f1b-8428909c16cd admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.677648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.680933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 857/1716] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:21.681045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.681045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.681045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] PATH_INFO: `/v3/projects/c031bf3c14104617939b3578b508594b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.681274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.681442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c031bf3c14104617939b3578b508594b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.681517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.682190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.692891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2c0a1-f36f-45da-9a6b-ddfcbdb3b4ea None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.697697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.698854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.699175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.699389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.700065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.701683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.715696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 858/1717] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:21.715696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.716047 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.772786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166b1e0-ef2f-419c-98ba-f9be30a3629e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.777102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.777700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.777977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.778247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.786939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.788438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.793167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:21.797762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 859/1718] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:21.798662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.809368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c031bf3c14104617939b3578b508594b', 'request_id': 'req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.809581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf84e-08e3-4bc2-bf04-93489b236bf2 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.815365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c031bf3c14104617939b3578b508594b', 'request_id': 'req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.822663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c031bf3c14104617939b3578b508594b', 'request_id': 'req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.867908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.868970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.868970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.869183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.869825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.871533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.876579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:21.885243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 860/1719] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:21.889906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:21.899625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf81ccb-8570-4893-9548-23ad6ebb4e43 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:21.905445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaab233-83de-4b90-a0ac-9968d060159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:21.905556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaab233-83de-4b90-a0ac-9968d060159c admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.905789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaab233-83de-4b90-a0ac-9968d060159c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.906061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaab233-83de-4b90-a0ac-9968d060159c admin admin] PATH_INFO: `/v3/projects/201f1ed3d5194c67b15ddda32b8b05ca/users/b027365d5bd240b38938ac15d6a5c129/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.920263 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] The token cache is being invalidate because project c031bf3c14104617939b3578b508594b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:21.920651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.921388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ab2a46b9-2fe1-4878-bbbb-9ce1fc983f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:21.922466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaab233-83de-4b90-a0ac-9968d060159c admin admin] RBAC: Authorizing `identity:create_grant(project_id=201f1ed3d5194c67b15ddda32b8b05ca, user_id=b027365d5bd240b38938ac15d6a5c129, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:21.923004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 860/1720] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:21 2022] DELETE /identity/v3/projects/c031bf3c14104617939b3578b508594b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:21.925808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaab233-83de-4b90-a0ac-9968d060159c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:21.928561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ca21e-d5f3-4158-9b18-09afc4d94f6f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.929089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ca21e-d5f3-4158-9b18-09afc4d94f6f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.929546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ca21e-d5f3-4158-9b18-09afc4d94f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.949744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-670ca21e-d5f3-4158-9b18-09afc4d94f6f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:21.950926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-670ca21e-d5f3-4158-9b18-09afc4d94f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.958852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 861/1721] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:21.966886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:21.967193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:21.967462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:21.986728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:21.988061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:21.990679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 861/1722] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:21 2022] PUT /identity/v3/projects/201f1ed3d5194c67b15ddda32b8b05ca/users/b027365d5bd240b38938ac15d6a5c129/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:22.001500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.001988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.002371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.045448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] MFA Rules not processed for user `b027365d5bd240b38938ac15d6a5c129`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:22.047575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.074862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 862/1723] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:22.079338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.103778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c40ff-c2b8-45bc-931f-fa381050bcd1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.116880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.117452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.117735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.118006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.119212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:22.119212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.121143 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.137871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 862/1724] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:22.149257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 863/1725] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:09:22.156931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.157656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.160941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.171434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9b9f-5f4c-4861-9f93-f224cbf7e428 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.177067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.178081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.178618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.178899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.179567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.181700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.229578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7aaf3-958d-4c40-ac49-928745f10475 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.236547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.237317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.237716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.237867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.248652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.250153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.261715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 864/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:22.262548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.264533 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.283892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472580-857c-45f8-bc93-13a61bbef1cd service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.291892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.292605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.295901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.295901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] PATH_INFO: `/v3/projects/4e01433c0cd24e698630188f5d55dae6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.296132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 863/1727] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:22 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:22.296277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e01433c0cd24e698630188f5d55dae6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.296371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.296684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.297028 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.362802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e484cbc-df22-4aee-88a9-d789ff50550e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.369298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.369298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.369298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.369298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] PATH_INFO: `/v3/users/b027365d5bd240b38938ac15d6a5c129` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.369829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b027365d5bd240b38938ac15d6a5c129)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.372397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.406832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:22.419455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e01433c0cd24e698630188f5d55dae6', 'request_id': 'req-866878cd-d803-40f5-acb6-460850f15bdf'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.425115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e01433c0cd24e698630188f5d55dae6', 'request_id': 'req-866878cd-d803-40f5-acb6-460850f15bdf'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.431171 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e01433c0cd24e698630188f5d55dae6', 'request_id': 'req-866878cd-d803-40f5-acb6-460850f15bdf'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.514508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] The token cache is being invalidate because project 4e01433c0cd24e698630188f5d55dae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:22.514805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.515472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.517010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 865/1728] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:22 2022] DELETE /identity/v3/projects/4e01433c0cd24e698630188f5d55dae6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:22.517914 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.519202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.578224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:22.580824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b027365d5bd240b38938ac15d6a5c129'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.581420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866878cd-d803-40f5-acb6-460850f15bdf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.581884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b027365d5bd240b38938ac15d6a5c129'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.586307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de549e89-77c4-4e98-85c4-eb854c0e40f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.586917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de549e89-77c4-4e98-85c4-eb854c0e40f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.587197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de549e89-77c4-4e98-85c4-eb854c0e40f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.587469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de549e89-77c4-4e98-85c4-eb854c0e40f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.588128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de549e89-77c4-4e98-85c4-eb854c0e40f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.590037 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de549e89-77c4-4e98-85c4-eb854c0e40f6 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.674656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b027365d5bd240b38938ac15d6a5c129'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.687909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b027365d5bd240b38938ac15d6a5c129'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.688598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b027365d5bd240b38938ac15d6a5c129'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.700778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 864/1729] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:22 2022] DELETE /identity/v3/users/b027365d5bd240b38938ac15d6a5c129 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:22.702405 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.707021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.711219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 866/1730] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:22 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:22.712565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.712852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.713126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.734691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:22.735897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.778483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c4182-ea5e-4b35-8bff-a93aa00fd8b6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.783743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.784414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.784697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.784969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] PATH_INFO: `/v3/users/4622da02634444749e8d1010b64f18fc` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.786350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4622da02634444749e8d1010b64f18fc)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.789306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.804781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 867/1731] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:22.805887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.807681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.821211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdb844-d6ae-4736-b715-639f541fb79f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.826032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.826778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.827072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.827346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.828146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.829752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.843682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 868/1732] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:22.844643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.848187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:22.892243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:22.905724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b0ed-063d-4940-a3cd-bb0ce417cc9c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.909366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.909968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.910302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.910578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.920254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.921662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.933159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 869/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:22.933159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.940933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1211e-af2f-4f89-992d-d65b7a9ced44 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.945690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.946254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.946534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.946803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:22.947421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:22.949054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:22.959383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 870/1734] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:22.963216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:22.974296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee9c5-f3c9-49f8-b41f-3281bd0e168a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:22.981076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4622da02634444749e8d1010b64f18fc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.981275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:22.982179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:22.982300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4622da02634444749e8d1010b64f18fc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:22.982393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:22.982702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] PATH_INFO: `/v3/projects/a6dbd14ffb7742a9889253d784fddb5a/users/23f9a7b5edaa4405a768b0a20199af48/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.008847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dbd14ffb7742a9889253d784fddb5a, user_id=23f9a7b5edaa4405a768b0a20199af48, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.012912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:23.035167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4622da02634444749e8d1010b64f18fc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.049945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4622da02634444749e8d1010b64f18fc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.050729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5688e74e-e70c-44a6-8392-1bea601d61e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4622da02634444749e8d1010b64f18fc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.063270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 865/1735] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:22 2022] DELETE /identity/v3/users/4622da02634444749e8d1010b64f18fc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:23.110058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 871/1736] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:22 2022] PUT /identity/v3/projects/a6dbd14ffb7742a9889253d784fddb5a/users/23f9a7b5edaa4405a768b0a20199af48/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:23.121764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.122423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.123795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.191958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] MFA Rules not processed for user `23f9a7b5edaa4405a768b0a20199af48`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:23.194440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:23.240800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:23.312188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 866/1737] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:23.313874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:23.316260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:23.418294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f658f7-6581-4bc3-9fb5-1a41f36837d1 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:23.423383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:23.424108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.424466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.424816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.425469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.427420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:23.516278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:23.518269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:23.535251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69986fbe-369e-475b-a0e0-657d6a8652a7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:23.548354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:23.549198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.549493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.549870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] PATH_INFO: `/v3/projects/201f1ed3d5194c67b15ddda32b8b05ca` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.551006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201f1ed3d5194c67b15ddda32b8b05ca)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.551627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 872/1738] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:23 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:23.555929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:23.556701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:23.646946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224cd6-eca8-4b4c-acd8-ee1a23ec42ed admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:23.652057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:23.652755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.653066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.653341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.654005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.656016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:23.711077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201f1ed3d5194c67b15ddda32b8b05ca', 'request_id': 'req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.718243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 873/1739] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:23 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:23.718243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:23.718514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:23.730080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90aa40-5c6c-4ac3-be6c-7c867b049883 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:23.750696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:23.783775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201f1ed3d5194c67b15ddda32b8b05ca', 'request_id': 'req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.789162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:23.789991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.790271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.790545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] PATH_INFO: `/v3/projects/f8952e8a7cc94f2ca3ab880eeb5f83c6` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.791518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201f1ed3d5194c67b15ddda32b8b05ca', 'request_id': 'req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.792236 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8952e8a7cc94f2ca3ab880eeb5f83c6)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.793748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:23.803625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] The token cache is being invalidate because project 201f1ed3d5194c67b15ddda32b8b05ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:23.804223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.805072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:23.806711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 867/1740] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:23 2022] DELETE /identity/v3/projects/201f1ed3d5194c67b15ddda32b8b05ca => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:23.808334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:23.811836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:23.825283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ace8-4b4e-410b-8144-b5bf5cb0cad8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:23.830009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33c196-1c0d-4826-85a0-0cf1a0e4b696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:23.830615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33c196-1c0d-4826-85a0-0cf1a0e4b696 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.830891 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33c196-1c0d-4826-85a0-0cf1a0e4b696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.831159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33c196-1c0d-4826-85a0-0cf1a0e4b696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.831790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33c196-1c0d-4826-85a0-0cf1a0e4b696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.833626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33c196-1c0d-4826-85a0-0cf1a0e4b696 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:23.844046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 868/1741] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:23.845851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.845851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.845851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.867241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:23.869090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:23.880812 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 869/1742] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:23.881981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:23.894432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25c8e8-55be-44f5-8615-cba052381706 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:23.899293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427dae8-9138-4eec-85db-c783944eee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:23.899958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427dae8-9138-4eec-85db-c783944eee4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:23.900419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427dae8-9138-4eec-85db-c783944eee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:23.900527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427dae8-9138-4eec-85db-c783944eee4d admin admin] PATH_INFO: `/v3/projects/8f9b6604adc84b74b57545a69cf88249/users/ae9ecd1e0b304918974fadd92a8bbf64/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:23.916448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427dae8-9138-4eec-85db-c783944eee4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9b6604adc84b74b57545a69cf88249, user_id=ae9ecd1e0b304918974fadd92a8bbf64, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:23.919632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427dae8-9138-4eec-85db-c783944eee4d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.059546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 870/1743] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:23 2022] PUT /identity/v3/projects/8f9b6604adc84b74b57545a69cf88249/users/ae9ecd1e0b304918974fadd92a8bbf64/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:24.063879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.064287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.064643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.068736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8952e8a7cc94f2ca3ab880eeb5f83c6', 'request_id': 'req-6d7b3b50-04b5-482e-8048-4c992aa85717'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:24.075315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8952e8a7cc94f2ca3ab880eeb5f83c6', 'request_id': 'req-6d7b3b50-04b5-482e-8048-4c992aa85717'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:24.082429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8952e8a7cc94f2ca3ab880eeb5f83c6', 'request_id': 'req-6d7b3b50-04b5-482e-8048-4c992aa85717'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:24.086829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:24.088083 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.149709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 871/1744] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:24.150944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.160699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5338f-bf8d-4698-b6b9-c8621bed6736 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.165384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.165987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.166260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.166532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.167169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.169365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.174914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:24.179803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 872/1745] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:24.180817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.191575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfbc2e-69e5-4af0-abd4-baf983571bf0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.195791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] The token cache is being invalidate because project f8952e8a7cc94f2ca3ab880eeb5f83c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:24.196129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:24.196272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.196764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:24.196862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.197129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.197388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.198013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:24.198240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 874/1746] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:23 2022] DELETE /identity/v3/projects/f8952e8a7cc94f2ca3ab880eeb5f83c6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:24.198327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.198989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.199857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90265e1e-a038-4016-b306-5b5cabab483e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.200329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.215621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 873/1747] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:24.215800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b3b50-04b5-482e-8048-4c992aa85717 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.215966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbd61-f504-4a57-b753-ad21c0b107d6 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.215966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbd61-f504-4a57-b753-ad21c0b107d6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.215966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbd61-f504-4a57-b753-ad21c0b107d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.223120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.223120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.223120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.223120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] PATH_INFO: `/v3/projects/8f9b6604adc84b74b57545a69cf88249/users/ae9ecd1e0b304918974fadd92a8bbf64/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.223482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9b6604adc84b74b57545a69cf88249, user_id=ae9ecd1e0b304918974fadd92a8bbf64, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.224674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.234111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-676dbd61-f504-4a57-b753-ad21c0b107d6 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:24.235519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-676dbd61-f504-4a57-b753-ad21c0b107d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.236010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 875/1748] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:24 2022] PUT /identity/v3/projects/8f9b6604adc84b74b57545a69cf88249/users/ae9ecd1e0b304918974fadd92a8bbf64/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:24.244728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.244728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.294137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 874/1749] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:24.295257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.295541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.295816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.305597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a140787-584c-4b2f-822e-0b0926b4f521 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.310514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d1d89-5eca-4242-adf5-95d1254ffce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.311293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d1d89-5eca-4242-adf5-95d1254ffce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.314819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d1d89-5eca-4242-adf5-95d1254ffce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.314819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d1d89-5eca-4242-adf5-95d1254ffce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.314819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d1d89-5eca-4242-adf5-95d1254ffce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.314819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d1d89-5eca-4242-adf5-95d1254ffce8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.323751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] MFA Rules not processed for user `ae9ecd1e0b304918974fadd92a8bbf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:24.324993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.372947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 876/1750] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:24.374325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.374617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.374890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.393660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 875/1751] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:24.393838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:24.394538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.394910 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.407915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bba54d-0647-460e-aef8-b212d28892ef None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.416713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:24.460407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 877/1752] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:24.461546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.463781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.475744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.475744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.475744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.475744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.476192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.476499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.522802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec185d6-79f6-4622-9332-bd32c9d31e9a None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.526636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.527771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.528077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.528352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.549874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 876/1753] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:24.550698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.555989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:24.564059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea308aa3-ea01-4667-b881-49d4beaf18e2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.569200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.569808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.570239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.570534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.571161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:24.571458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.573087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.586314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 877/1754] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:24.587256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.598109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cc348-0faf-42ae-9513-56b8921d831f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.603125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.603712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.603998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.604312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.604981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.605135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.606568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.606973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.616479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 878/1755] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:24.617088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.617782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 878/1756] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:24.624926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.633187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639ee79-56aa-4c87-85ff-3f9c4b0b4594 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.637964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a89115-cec9-4baa-a43a-e4340544b291 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.638902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.640010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.640010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.640279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.640922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.642858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.644103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.644951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.645544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.646543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.647213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.649852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.664064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 879/1757] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:24.665916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.676918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151ab7f-7913-4e23-90b1-f1d0f1798c04 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.681540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0dd57-e70d-44b3-be8a-b2a51d230913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.682197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0dd57-e70d-44b3-be8a-b2a51d230913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.682473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0dd57-e70d-44b3-be8a-b2a51d230913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.682745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0dd57-e70d-44b3-be8a-b2a51d230913 admin admin] PATH_INFO: `/v3/projects/4abcd297b89c4351a268c6c4e972de80/users/7fb0eb26fe604c02a5db5a114202c127/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.699276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0dd57-e70d-44b3-be8a-b2a51d230913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abcd297b89c4351a268c6c4e972de80, user_id=7fb0eb26fe604c02a5db5a114202c127, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.702309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0dd57-e70d-44b3-be8a-b2a51d230913 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.728190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 879/1758] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:24.729509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.789237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 880/1759] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:24 2022] PUT /identity/v3/projects/4abcd297b89c4351a268c6c4e972de80/users/7fb0eb26fe604c02a5db5a114202c127/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:24.790789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:24.800784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.800917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.801230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.801753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfb6ca-b913-4e76-be06-60289e5c2db9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:24.837080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] MFA Rules not processed for user `7fb0eb26fe604c02a5db5a114202c127`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:24.838432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:24.877302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:24.878054 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:24.878609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:24.878950 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:24.880884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:24.884124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:24.934589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 881/1760] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:24.946120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:24.950148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:25.031159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d61f-b02a-4ae9-8853-2eb4dfbdfb01 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.039595 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 880/1761] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:24 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:25.039780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.039780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.039780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.039780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.040743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.051068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.053971 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.063301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2c73c-12bd-480a-a397-b39abcb3b402 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.073154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 882/1762] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:25.074252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.074965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.075391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.075735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.076408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.077983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.095423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 881/1763] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:25.096668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.108658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959667fd-863a-4e38-b750-25bd87ba36fa service glance] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.114378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144829a8-beb5-496f-ab90-1fd064dda3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.119077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144829a8-beb5-496f-ab90-1fd064dda3ac admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.119077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144829a8-beb5-496f-ab90-1fd064dda3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.119077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144829a8-beb5-496f-ab90-1fd064dda3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.119077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144829a8-beb5-496f-ab90-1fd064dda3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.119077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144829a8-beb5-496f-ab90-1fd064dda3ac admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.128602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 883/1764] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:25.132842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.146803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62fbf3-b16c-4ff6-8c78-aa7ca6a0e45e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.152064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.152806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.153195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.153495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] PATH_INFO: `/v3/projects/373c7390efea4064b3e3aea1e90423ca/users/9aaa096ece7849c0965c69279300688e/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.170987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] RBAC: Authorizing `identity:create_grant(project_id=373c7390efea4064b3e3aea1e90423ca, user_id=9aaa096ece7849c0965c69279300688e, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.173721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.275010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 882/1765] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:25 2022] PUT /identity/v3/projects/373c7390efea4064b3e3aea1e90423ca/users/9aaa096ece7849c0965c69279300688e/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:25.287243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.287692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.287784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.296573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:25.318605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] MFA Rules not processed for user `9aaa096ece7849c0965c69279300688e`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:25.319914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:25.388517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 884/1766] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:25.393464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.459444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514786-0b81-46f2-934e-4075248d621b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.464354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.465162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.465451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.465729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.466363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.468283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.564588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 883/1767] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:25 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:25.569106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.579676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b03e3-22fe-4b39-94a9-8709bd5606c0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.648229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.648958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.649647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.650110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.650762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.653039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.755160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 885/1768] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:25 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:25.759398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.769584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1cc56-e30a-4070-bf8d-102b6ccaefee admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.774080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.774668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.774944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.775214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.775845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.778157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.788509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 884/1769] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:25.794385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.804737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d3f6c-2b8c-4d80-b6bc-9d2392c786eb admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.809422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.810080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.810432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.810652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] PATH_INFO: `/v3/projects/f142cf66735e42d9a642c50424bab23e/users/8b8fa9ffe21e4abb8b94f14ba9ada651/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.825730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f142cf66735e42d9a642c50424bab23e, user_id=8b8fa9ffe21e4abb8b94f14ba9ada651, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:25.828542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:25.897659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.901596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:25.909762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 886/1770] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:25 2022] PUT /identity/v3/projects/f142cf66735e42d9a642c50424bab23e/users/8b8fa9ffe21e4abb8b94f14ba9ada651/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:25.913918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:25.915435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de99b71-fed9-461d-826c-1c60c21ecc36 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:25.945601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:25.996275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:25.997273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:25.997391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:25.998406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] PATH_INFO: `/v3/users/ef45c86c5a8e46379b116ab095d2f0ef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:25.999166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef45c86c5a8e46379b116ab095d2f0ef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.001322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.009746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4754473-d9bf-4bea-a847-72621308aa2e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.015974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.016963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.017385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.017798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.018592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.020328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.022322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:26.032799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 887/1771] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:26.036741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.052852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce7be-067f-4a7c-b2f1-69340566f053 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.059455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6e98-21fe-4a92-bee9-c068b64aea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.060181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6e98-21fe-4a92-bee9-c068b64aea8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.060463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6e98-21fe-4a92-bee9-c068b64aea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.060739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6e98-21fe-4a92-bee9-c068b64aea8f admin admin] PATH_INFO: `/v3/projects/f142cf66735e42d9a642c50424bab23e/users/8b8fa9ffe21e4abb8b94f14ba9ada651/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.062804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b6e98-21fe-4a92-bee9-c068b64aea8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f142cf66735e42d9a642c50424bab23e, user_id=8b8fa9ffe21e4abb8b94f14ba9ada651, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.066353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b6e98-21fe-4a92-bee9-c068b64aea8f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.084695 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 888/1772] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:26 2022] PUT /identity/v3/projects/f142cf66735e42d9a642c50424bab23e/users/8b8fa9ffe21e4abb8b94f14ba9ada651/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:26.094770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.095060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.095337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.128004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] MFA Rules not processed for user `8b8fa9ffe21e4abb8b94f14ba9ada651`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:26.129284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.196339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 889/1773] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:26.231392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.243004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef45c86c5a8e46379b116ab095d2f0ef'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.295142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3db0e-11ce-4d13-a8cc-496fb492cf49 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.299132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17513dbc-c49f-4d7c-a103-44e68595d670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.299736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17513dbc-c49f-4d7c-a103-44e68595d670 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.300015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17513dbc-c49f-4d7c-a103-44e68595d670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.300315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17513dbc-c49f-4d7c-a103-44e68595d670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.309992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17513dbc-c49f-4d7c-a103-44e68595d670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.311441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17513dbc-c49f-4d7c-a103-44e68595d670 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.321049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef45c86c5a8e46379b116ab095d2f0ef'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.326598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 890/1774] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:26.326717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef45c86c5a8e46379b116ab095d2f0ef'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.326717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef45c86c5a8e46379b116ab095d2f0ef'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.338080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef45c86c5a8e46379b116ab095d2f0ef'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.359467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 885/1775] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:25 2022] DELETE /identity/v3/users/ef45c86c5a8e46379b116ab095d2f0ef => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:26.363780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.364213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.364419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.385002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:26.386872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.390570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.392133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.451309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 891/1776] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:26.458559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d64b63-4f26-49b6-97eb-675231bd01d4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.461535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.462985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.463175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.463813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.464119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.464432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] PATH_INFO: `/v3/users/9aaa096ece7849c0965c69279300688e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.465558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aaa096ece7849c0965c69279300688e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.467466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.477149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:26.525736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56e18f-e7a8-413e-a5ee-1868776d8fe6 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.529267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.529869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.530144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.530412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.539111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.540564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.549785 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 892/1777] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:26.550637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.552009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.564444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe4c92-3a5c-4ad9-bfc8-5a8508e4af14 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.569255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.569864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.570164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.570415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] PATH_INFO: `/v3/users/23f9a7b5edaa4405a768b0a20199af48` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.571523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f9a7b5edaa4405a768b0a20199af48)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:26.573425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:26.671180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aaa096ece7849c0965c69279300688e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.673849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:26.733959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aaa096ece7849c0965c69279300688e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.735085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aaa096ece7849c0965c69279300688e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.735923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aaa096ece7849c0965c69279300688e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.748643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f9a7b5edaa4405a768b0a20199af48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.749917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f9a7b5edaa4405a768b0a20199af48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.751854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-444179d7-a123-4051-a209-469c7bdfda0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aaa096ece7849c0965c69279300688e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.774319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 886/1778] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:26 2022] DELETE /identity/v3/users/9aaa096ece7849c0965c69279300688e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:26.778869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.779580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.779580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.800102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:26.801325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.839399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f9a7b5edaa4405a768b0a20199af48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.854530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f9a7b5edaa4405a768b0a20199af48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.855784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4d644f74-88d6-4662-b0ae-4d720360dd30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f9a7b5edaa4405a768b0a20199af48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:26.867847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 887/1779] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:26.879478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.880904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.883116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 893/1780] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:26 2022] DELETE /identity/v3/users/23f9a7b5edaa4405a768b0a20199af48 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:26.887448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.887734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.888012 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.907076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:26.908242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.939395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aeaa7-5b7f-42c1-bdc6-e295d752c3ab None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:26.978170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 894/1781] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:26.981224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:26.983365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:26.994860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:26.996251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:26.996528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:26.996841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:26.996942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca780f1-5b22-43ef-9efc-507be2e383ce None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:27.001624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:27.002286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:27.002568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:27.003033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] PATH_INFO: `/v3/projects/a9e6089ccc5246b4ba26b2282c1c3ac1` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:27.004664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e6089ccc5246b4ba26b2282c1c3ac1)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.006631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.008449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.008559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.020163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 888/1782] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:27.024991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:27.026315 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:27.031981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:27.042076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e404c-75e2-4ba6-8e60-5eb1ff2608d7 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:27.046522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:27.047125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:27.047409 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:27.047679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:27.059953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.061498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.072361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 889/1783] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:27.134882 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e6089ccc5246b4ba26b2282c1c3ac1', 'request_id': 'req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.189342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e6089ccc5246b4ba26b2282c1c3ac1', 'request_id': 'req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.196362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e6089ccc5246b4ba26b2282c1c3ac1', 'request_id': 'req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.208375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] The token cache is being invalidate because project a9e6089ccc5246b4ba26b2282c1c3ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:27.208668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.209373 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.213855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 895/1784] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:26 2022] DELETE /identity/v3/projects/a9e6089ccc5246b4ba26b2282c1c3ac1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:27.487176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:27.488762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:27.502962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6395-5487-4230-9058-520c03f9e386 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:27.513264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:27.563011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:27.564843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:27.565142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:27.565421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] PATH_INFO: `/v3/projects/373c7390efea4064b3e3aea1e90423ca` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:27.566852 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=373c7390efea4064b3e3aea1e90423ca)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.568786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.575299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:27.576898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:27.590948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c37be-f451-43ce-9fcb-1b7000cbc83b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:27.596418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:27.597069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:27.597348 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:27.597686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] PATH_INFO: `/v3/projects/a6dbd14ffb7742a9889253d784fddb5a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:27.598948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dbd14ffb7742a9889253d784fddb5a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.600897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.685982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373c7390efea4064b3e3aea1e90423ca', 'request_id': 'req-ec996937-bd93-4270-a22f-c6724a439611'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.691544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373c7390efea4064b3e3aea1e90423ca', 'request_id': 'req-ec996937-bd93-4270-a22f-c6724a439611'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.697767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373c7390efea4064b3e3aea1e90423ca', 'request_id': 'req-ec996937-bd93-4270-a22f-c6724a439611'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.715835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dbd14ffb7742a9889253d784fddb5a', 'request_id': 'req-dc033694-db79-4542-9410-236fea7cded6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.788319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] The token cache is being invalidate because project 373c7390efea4064b3e3aea1e90423ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:27.788819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.789226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.790711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 890/1785] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:27 2022] DELETE /identity/v3/projects/373c7390efea4064b3e3aea1e90423ca => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:27.794449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:27.795825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:27.851527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dbd14ffb7742a9889253d784fddb5a', 'request_id': 'req-dc033694-db79-4542-9410-236fea7cded6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.856918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec996937-bd93-4270-a22f-c6724a439611 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:27.857871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dbd14ffb7742a9889253d784fddb5a', 'request_id': 'req-dc033694-db79-4542-9410-236fea7cded6'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.879352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] The token cache is being invalidate because project a6dbd14ffb7742a9889253d784fddb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:27.879648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.880337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:27.881895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 896/1786] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:27 2022] DELETE /identity/v3/projects/a6dbd14ffb7742a9889253d784fddb5a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:27.885689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:27.887569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:27.914572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:27.915207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:27.915488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:27.915759 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] PATH_INFO: `/v3/users/8b8fa9ffe21e4abb8b94f14ba9ada651` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:27.916923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b8fa9ffe21e4abb8b94f14ba9ada651)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.918949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.956061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc033694-db79-4542-9410-236fea7cded6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:27.960970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:27.961607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:27.961887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:27.962158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] PATH_INFO: `/v3/users/ae9ecd1e0b304918974fadd92a8bbf64` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:27.963296 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9ecd1e0b304918974fadd92a8bbf64)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:27.965293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:27.995181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:28.083224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b8fa9ffe21e4abb8b94f14ba9ada651'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.085662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:28.095145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9ecd1e0b304918974fadd92a8bbf64'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.095811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9ecd1e0b304918974fadd92a8bbf64'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.160637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b8fa9ffe21e4abb8b94f14ba9ada651'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.161285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b8fa9ffe21e4abb8b94f14ba9ada651'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.161931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b8fa9ffe21e4abb8b94f14ba9ada651'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.174494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b8fa9ffe21e4abb8b94f14ba9ada651'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.186842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 891/1787] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:27 2022] DELETE /identity/v3/users/8b8fa9ffe21e4abb8b94f14ba9ada651 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:28.235082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9ecd1e0b304918974fadd92a8bbf64'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.248873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9ecd1e0b304918974fadd92a8bbf64'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.249548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9ecd1e0b304918974fadd92a8bbf64'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.262479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 897/1788] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:27 2022] DELETE /identity/v3/users/ae9ecd1e0b304918974fadd92a8bbf64 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:28.559261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:28.560773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:28.638695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79799d02-19c9-402e-8b3e-fc06cb1c4c16 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:28.698770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:28.699483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:28.699835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:28.700211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] PATH_INFO: `/v3/projects/f142cf66735e42d9a642c50424bab23e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:28.701504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f142cf66735e42d9a642c50424bab23e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:28.703605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:28.726067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:28.744598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:28.745958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:28.766056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8460bc-f054-45e5-bbbf-3560e96334f8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:28.773613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:28.775639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:28.776101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:28.776432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] PATH_INFO: `/v3/projects/8f9b6604adc84b74b57545a69cf88249` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:28.778121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9b6604adc84b74b57545a69cf88249)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:28.780018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:28.874700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f142cf66735e42d9a642c50424bab23e', 'request_id': 'req-341b9df1-d573-4b20-92c1-d0a00e1b721b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.880487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f142cf66735e42d9a642c50424bab23e', 'request_id': 'req-341b9df1-d573-4b20-92c1-d0a00e1b721b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.886800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f142cf66735e42d9a642c50424bab23e', 'request_id': 'req-341b9df1-d573-4b20-92c1-d0a00e1b721b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:28.980549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9b6604adc84b74b57545a69cf88249', 'request_id': 'req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.045826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] The token cache is being invalidate because project f142cf66735e42d9a642c50424bab23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:29.046114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.046781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-341b9df1-d573-4b20-92c1-d0a00e1b721b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.048340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 892/1789] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:28 2022] DELETE /identity/v3/projects/f142cf66735e42d9a642c50424bab23e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:29.053052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b763-9a00-4bfe-be6b-7e57d573c103 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.053338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b763-9a00-4bfe-be6b-7e57d573c103 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.053613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b763-9a00-4bfe-be6b-7e57d573c103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.074677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e3d4b763-9a00-4bfe-be6b-7e57d573c103 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:29.076135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e3d4b763-9a00-4bfe-be6b-7e57d573c103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:29.139256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 893/1790] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:29.146845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.147130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.147619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.148893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9b6604adc84b74b57545a69cf88249', 'request_id': 'req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.156389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9b6604adc84b74b57545a69cf88249', 'request_id': 'req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.170339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:29.170858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] The token cache is being invalidate because project 8f9b6604adc84b74b57545a69cf88249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:29.171148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.171520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:29.171990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-38a23ef7-ba01-4619-b65d-9cf32d0b19e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:29.173646 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 898/1791] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:28 2022] DELETE /identity/v3/projects/8f9b6604adc84b74b57545a69cf88249 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:29.180268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d33bb-068c-4583-9e58-bb5b30282e1b None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.180558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d33bb-068c-4583-9e58-bb5b30282e1b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.181172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d33bb-068c-4583-9e58-bb5b30282e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.183591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 894/1792] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:29.187927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.199206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5f410-47f5-41f4-8ff8-2e10cd96e5fd None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.204536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.205224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.205572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.205751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-4d8d33bb-068c-4583-9e58-bb5b30282e1b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:29.205971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.206688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:29.206969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4d8d33bb-068c-4583-9e58-bb5b30282e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:29.207095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.209073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.218017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 899/1793] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:29.222682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 895/1794] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:29.225880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.226172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.226453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.227516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.238399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6b026-61a2-49b9-9ca8-9a56ff06bd13 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.243360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.243975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.244311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.244583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.245217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.246338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:29.247115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.247584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:29.256111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 900/1795] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:29.260409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.270937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22f73f-d85f-4c84-bf12-2e0f49ecfb58 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.275567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.276211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.276493 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.277393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.278089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:29.278393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.280164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.296334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 901/1796] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:29.298202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.308768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d3c2c-557e-40ad-94d7-2043950ab925 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.313590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.314189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.314469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.314741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.315366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.317629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.350021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 896/1797] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:29.354374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.408158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 902/1798] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:29.414834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.443666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d9bb1-5145-4b88-bbb6-5b026e3e1fb5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.478653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:29.493595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196695b-0dfa-4582-a4a3-cb4e7584dabe admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.498873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.499482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.499765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.500093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.500737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.502692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.505877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.506498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.506775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.507046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.507676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.510100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.515759 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:29.589068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 903/1799] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:29.593920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.605402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d378-1638-4a51-bd68-b4a86893394c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.611328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.612543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.613141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.613419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.614246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.615843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.623149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 897/1800] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:29.625889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.627912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 904/1801] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:29.637717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc48e76-c411-406d-ae32-08ff9a372f2c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.638915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.642612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.643333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.643680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.644058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.644794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.646371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.648917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee17199-07e6-4669-8d04-700349965e49 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.656834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 898/1802] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:29.658280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.658919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.659194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.659471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.660199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.661942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.662063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.672511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76027f-6e27-4908-a08e-7c2605f85edf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.674634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 905/1803] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:29.678514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.679138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.681812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.681812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.681812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.682062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.682689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.692937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50d3d6-e948-4c02-b4e5-5f0c6ce705da admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.696643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 899/1804] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:29.697495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.702191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f95144-621e-4663-9cc8-23475484a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.702851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95144-621e-4663-9cc8-23475484a9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.708623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95144-621e-4663-9cc8-23475484a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.708623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95144-621e-4663-9cc8-23475484a9b4 admin admin] PATH_INFO: `/v3/projects/f65c6ce636154468858e7ac486321a64/users/102187101b8a4d34928e3d7afbe2df46/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.709627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8901-982a-4590-9d1d-9a8dbbf7d6ba admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:29.712123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a045f-b7ce-4455-8caf-880a7385c874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:29.712770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a045f-b7ce-4455-8caf-880a7385c874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.713099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a045f-b7ce-4455-8caf-880a7385c874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.713379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a045f-b7ce-4455-8caf-880a7385c874 admin admin] PATH_INFO: `/v3/projects/f9e048e32c184699b833b7d03b49d3bf/users/6abfa7dac7884aacba26fd761620d5a5/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.718276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f95144-621e-4663-9cc8-23475484a9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65c6ce636154468858e7ac486321a64, user_id=102187101b8a4d34928e3d7afbe2df46, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.720956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f95144-621e-4663-9cc8-23475484a9b4 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.728674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a045f-b7ce-4455-8caf-880a7385c874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e048e32c184699b833b7d03b49d3bf, user_id=6abfa7dac7884aacba26fd761620d5a5, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:29.731392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a045f-b7ce-4455-8caf-880a7385c874 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:29.834406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 906/1805] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:29 2022] PUT /identity/v3/projects/f65c6ce636154468858e7ac486321a64/users/102187101b8a4d34928e3d7afbe2df46/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:29.845861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.846151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.846432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.860554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 900/1806] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:29 2022] PUT /identity/v3/projects/f9e048e32c184699b833b7d03b49d3bf/users/6abfa7dac7884aacba26fd761620d5a5/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:29.871336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:29.871624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:29.871901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:29.883843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] MFA Rules not processed for user `102187101b8a4d34928e3d7afbe2df46`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:29.885127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:29.900109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] MFA Rules not processed for user `6abfa7dac7884aacba26fd761620d5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:29.901313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:29.953441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 907/1807] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:29.957952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.967761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 901/1808] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:29.978347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:29.980288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:30.019000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c4c17-95ca-4e7d-bd68-914925c71d21 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.023836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.024505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.024797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.025053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.025708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.027609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.036186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:30.046778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cd71a-4c32-4884-905f-1f517e1b7fa4 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.052432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.052432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.052432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.052432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.061982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.063441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.072386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 902/1809] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:30.139031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 908/1810] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:29 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:30.143221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:30.153595 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5bb25-48d6-43dc-af7d-e7be6ed5028b service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.212431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.212976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.213254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.213525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.214702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.216679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.320297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 903/1811] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:30 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:30.324212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:30.330751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:30.334552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7c7cf-e22a-4fa7-a5ce-05cc7f20a390 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.339817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.340446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.340726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.341000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.341641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.343213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.352838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 909/1812] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:30.357653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:30.367686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25681abe-5449-43c1-9eac-22a1b2366e62 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.372404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.373052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.373336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.373613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] PATH_INFO: `/v3/projects/e5567895c27c42e7aa82292b9e94a234/users/38168faa2ed34641991446cdd17fbf4c/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.388219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5567895c27c42e7aa82292b9e94a234, user_id=38168faa2ed34641991446cdd17fbf4c, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.390855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.463219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 904/1813] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:30 2022] PUT /identity/v3/projects/e5567895c27c42e7aa82292b9e94a234/users/38168faa2ed34641991446cdd17fbf4c/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:30.467161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:30.477292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61595d-f3ba-458d-b8b5-b4de8b9e0523 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.556866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2758dc-deb4-44d5-870d-98c41165f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.557479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2758dc-deb4-44d5-870d-98c41165f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.558312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2758dc-deb4-44d5-870d-98c41165f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.558626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2758dc-deb4-44d5-870d-98c41165f0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.559431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2758dc-deb4-44d5-870d-98c41165f0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.561081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2758dc-deb4-44d5-870d-98c41165f0e1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.571786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 910/1814] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:30.576598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:30.586865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42185c-4cf7-4b55-8357-8fe890fbe81a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.591472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.592147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.592428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.592693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] PATH_INFO: `/v3/projects/e5567895c27c42e7aa82292b9e94a234/users/38168faa2ed34641991446cdd17fbf4c/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.594584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5567895c27c42e7aa82292b9e94a234, user_id=38168faa2ed34641991446cdd17fbf4c, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.597250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.608485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 905/1815] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:30 2022] PUT /identity/v3/projects/e5567895c27c42e7aa82292b9e94a234/users/38168faa2ed34641991446cdd17fbf4c/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:30.618619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.619078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.619353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.647534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] MFA Rules not processed for user `38168faa2ed34641991446cdd17fbf4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:30.648798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:30.720951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 911/1816] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:30.733396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:30.790765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314aa4b4-f02e-448d-bf2c-ea750aa5cf95 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:30.795129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:30.795770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:30.796076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:30.796352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:30.805709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:30.807123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:30.818795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 906/1817] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:31.038132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:31.048762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e472862-eb04-4570-8c9d-5778f3b837e2 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:31.052371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc098b-5bf2-47b4-bdf4-c51c3a2d1464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:31.052969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc098b-5bf2-47b4-bdf4-c51c3a2d1464 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:31.056378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc098b-5bf2-47b4-bdf4-c51c3a2d1464 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:31.056378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc098b-5bf2-47b4-bdf4-c51c3a2d1464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:31.109704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc098b-5bf2-47b4-bdf4-c51c3a2d1464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:31.111152 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc098b-5bf2-47b4-bdf4-c51c3a2d1464 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:31.120233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 912/1818] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:31.561530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:31.613733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:31.634757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf00e25-f6bd-417a-a726-18e940ccc2fc service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:31.639929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:31.640672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:31.641032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:31.641727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] PATH_INFO: `/v3/users/102187101b8a4d34928e3d7afbe2df46` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:31.642937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=102187101b8a4d34928e3d7afbe2df46)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:31.644879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:31.831490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '102187101b8a4d34928e3d7afbe2df46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:31.913699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '102187101b8a4d34928e3d7afbe2df46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:31.914335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '102187101b8a4d34928e3d7afbe2df46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:31.914969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '102187101b8a4d34928e3d7afbe2df46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:31.927670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '102187101b8a4d34928e3d7afbe2df46'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:31.940438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 907/1819] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:31 2022] DELETE /identity/v3/users/102187101b8a4d34928e3d7afbe2df46 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:31.945162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:31.945453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:31.945733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:31.965382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:31.966568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:31.974479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:32.031945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 913/1820] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:32.041806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:32.043339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:32.099970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816bfb-1462-4629-9fd6-342eaea3ec23 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:32.103461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:32.104088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:32.104366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:32.104635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:32.113401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:32.115024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:32.125953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 908/1821] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:32.527714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:32.529200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:32.541457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd445-e0cf-46b0-9a2e-bf58cee40503 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:32.546734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:32.547372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:32.547651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:32.547917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] PATH_INFO: `/v3/projects/f65c6ce636154468858e7ac486321a64` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:32.549058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65c6ce636154468858e7ac486321a64)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:32.550932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:32.699359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65c6ce636154468858e7ac486321a64', 'request_id': 'req-26bd8fc5-f3d6-4d67-9137-9074bf135965'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:32.777465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65c6ce636154468858e7ac486321a64', 'request_id': 'req-26bd8fc5-f3d6-4d67-9137-9074bf135965'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:32.783581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65c6ce636154468858e7ac486321a64', 'request_id': 'req-26bd8fc5-f3d6-4d67-9137-9074bf135965'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:32.795541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] The token cache is being invalidate because project f65c6ce636154468858e7ac486321a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:32.795832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:32.796498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:32.798009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 914/1822] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:32 2022] DELETE /identity/v3/projects/f65c6ce636154468858e7ac486321a64 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:32.801972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:32.812637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444b92-ca4d-40ab-bd24-287ff8c454a2 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:32.879750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:32.879750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:32.879750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:32.879750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] PATH_INFO: `/v3/users/38168faa2ed34641991446cdd17fbf4c` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:32.880926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38168faa2ed34641991446cdd17fbf4c)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:32.883205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:32.890853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:33.105768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38168faa2ed34641991446cdd17fbf4c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:33.206416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38168faa2ed34641991446cdd17fbf4c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:33.207489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38168faa2ed34641991446cdd17fbf4c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:33.208337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38168faa2ed34641991446cdd17fbf4c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:33.221831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-2d95f2fe-538c-44ec-b7ac-40e8629cd05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38168faa2ed34641991446cdd17fbf4c'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:33.234335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 909/1823] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:32 2022] DELETE /identity/v3/users/38168faa2ed34641991446cdd17fbf4c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:33.696962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:33.697880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:33.723302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd8fc5-f3d6-4d67-9137-9074bf135965 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:33.824383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:33.825639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:33.825955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:33.826175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] PATH_INFO: `/v3/projects/e5567895c27c42e7aa82292b9e94a234` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:33.827389 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5567895c27c42e7aa82292b9e94a234)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:33.829327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:33.962562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:33.970171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5567895c27c42e7aa82292b9e94a234', 'request_id': 'req-ce7f0a76-9b96-4b09-8645-15b150e933e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:34.057580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5567895c27c42e7aa82292b9e94a234', 'request_id': 'req-ce7f0a76-9b96-4b09-8645-15b150e933e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:34.064616 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5567895c27c42e7aa82292b9e94a234', 'request_id': 'req-ce7f0a76-9b96-4b09-8645-15b150e933e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:34.077940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] The token cache is being invalidate because project e5567895c27c42e7aa82292b9e94a234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:34.078232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:34.079021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ce7f0a76-9b96-4b09-8645-15b150e933e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:34.081003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 915/1824] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:33 2022] DELETE /identity/v3/projects/e5567895c27c42e7aa82292b9e94a234 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:34.085956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.086418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.086539 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.108809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:34.110052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:34.178940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 910/1825] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:34.189784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.190068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.190375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.212109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:34.213347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:34.222033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 916/1826] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:34.226628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.238340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cad36c-6d29-4dfe-8c0b-f9beab7454d7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.245691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.245691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.246301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.247867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.248832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:34.249360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:34.251256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:34.266053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 911/1827] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:09:34.270735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.282014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1f8b5-163f-443d-92a1-a97dcdcb8241 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.287374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.288006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.288352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.288645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.289329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:34.291225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:34.373554 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 917/1828] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:34 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:34.377996 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.388042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eebe6f-fd93-43f0-ba8b-8006d71acee1 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.450020 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.450655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.450933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.451207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.451843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:34.453781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:34.539054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.539538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 912/1829] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:34 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:34.551512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a48505-c73d-40e9-b030-7767b802b91f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.556576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.557198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.557483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.558400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.558511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:34.560143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:34.577457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.578611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 918/1830] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:34.583912 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:34.594264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c6407-a8c0-4f00-974a-4eb3c99a5f61 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.605475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8393bde-370f-4b6b-bd9f-60a813c08c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.608763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8393bde-370f-4b6b-bd9f-60a813c08c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.608763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8393bde-370f-4b6b-bd9f-60a813c08c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.608763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8393bde-370f-4b6b-bd9f-60a813c08c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.609188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8393bde-370f-4b6b-bd9f-60a813c08c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:34.612519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8393bde-370f-4b6b-bd9f-60a813c08c25 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:34.635303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 913/1831] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:34.636638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.647950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f59af-2328-42f8-927f-f084d1e611d9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.654890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.655815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.656272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.656661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] PATH_INFO: `/v3/projects/2bae6fc6edb94abe808c54562d161029/users/c3a167c6dbb44c7a94536b7ff041ebe4/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.675551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bae6fc6edb94abe808c54562d161029, user_id=c3a167c6dbb44c7a94536b7ff041ebe4, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:34.678375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:34.784672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 919/1832] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:34 2022] PUT /identity/v3/projects/2bae6fc6edb94abe808c54562d161029/users/c3a167c6dbb44c7a94536b7ff041ebe4/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:34.794922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.795208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.795481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:34.823658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] MFA Rules not processed for user `c3a167c6dbb44c7a94536b7ff041ebe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:34.825484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:34.891558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 914/1833] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:34.903501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:34.905262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:34.964954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b8890-90ea-452b-acf9-af8532a6a9c9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:34.980509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:34.981542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:34.982019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:34.982306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.011438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:35.012820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:35.024200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 920/1834] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:35.045662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:35.124539 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1e190-0a2f-4fcd-a8b6-dc5a2a5015c9 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:35.129487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:35.130138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:35.130802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:35.131263 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] PATH_INFO: `/v3/users/c3a167c6dbb44c7a94536b7ff041ebe4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.133026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a167c6dbb44c7a94536b7ff041ebe4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:35.135180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:35.332187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:35.333598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:35.347778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9da877-fc6e-42c1-bc07-175dd99482b2 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:35.352580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:35.353251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:35.353533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:35.353807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] PATH_INFO: `/v3/users/6abfa7dac7884aacba26fd761620d5a5` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.354963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abfa7dac7884aacba26fd761620d5a5)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:35.356265 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a167c6dbb44c7a94536b7ff041ebe4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.356878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:35.473537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a167c6dbb44c7a94536b7ff041ebe4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.474739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a167c6dbb44c7a94536b7ff041ebe4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.476146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a167c6dbb44c7a94536b7ff041ebe4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.490849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3692122b-8347-4388-bf5f-23b2f194fd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a167c6dbb44c7a94536b7ff041ebe4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.500962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:35.505467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 915/1835] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:35 2022] DELETE /identity/v3/users/c3a167c6dbb44c7a94536b7ff041ebe4 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:35.510101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:35.510355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:35.510630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.530926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:35.532257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:35.590597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abfa7dac7884aacba26fd761620d5a5'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.591304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abfa7dac7884aacba26fd761620d5a5'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.641366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 916/1836] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:35.646707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abfa7dac7884aacba26fd761620d5a5'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.656802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:35.660085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:35.662830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:35.666963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abfa7dac7884aacba26fd761620d5a5'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.671295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-12be6c9b-580b-455e-ab90-97d0e99166a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abfa7dac7884aacba26fd761620d5a5'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:35.701596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 921/1837] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:35 2022] DELETE /identity/v3/users/6abfa7dac7884aacba26fd761620d5a5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:35.703801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:35.704193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:35.704550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.767813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:35.770982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:35.772762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71147ed2-4f94-4842-85dd-735bacc5a324 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:35.899070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 922/1838] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:35.907270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:35.908723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:35.910215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:35.910855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:35.911136 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:35.911937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.926173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4d3b2-f83a-4375-8d05-80be9835a025 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:35.928209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:35.929713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:35.939546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:35.940203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:35.940482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:35.941022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:35.946801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 917/1839] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:35.959294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:35.960764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:35.971884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 923/1840] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:36.568369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:36.570011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:36.584813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5692cd-1e45-4a9b-8cf7-a13662cf07d5 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:36.590026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:36.591014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:36.591310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:36.591647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] PATH_INFO: `/v3/projects/2bae6fc6edb94abe808c54562d161029` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:36.593145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bae6fc6edb94abe808c54562d161029)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:36.594944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:36.738638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bae6fc6edb94abe808c54562d161029', 'request_id': 'req-c0aef1dd-d777-4aef-9ec0-c47677fbb624'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:36.742599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:36.744724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bae6fc6edb94abe808c54562d161029', 'request_id': 'req-c0aef1dd-d777-4aef-9ec0-c47677fbb624'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:36.751798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bae6fc6edb94abe808c54562d161029', 'request_id': 'req-c0aef1dd-d777-4aef-9ec0-c47677fbb624'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:36.813368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:36.815064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:36.834487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed826e-5574-437a-81f8-9fd0464c1603 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:36.859880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] The token cache is being invalidate because project 2bae6fc6edb94abe808c54562d161029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:36.859880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:36.864381 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c0aef1dd-d777-4aef-9ec0-c47677fbb624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:36.866144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 918/1841] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:36 2022] DELETE /identity/v3/projects/2bae6fc6edb94abe808c54562d161029 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:36.870877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5867ba-9d90-4826-8d93-d46d3faa6d08 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:36.871875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5867ba-9d90-4826-8d93-d46d3faa6d08 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:36.872444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5867ba-9d90-4826-8d93-d46d3faa6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:36.916311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2c5867ba-9d90-4826-8d93-d46d3faa6d08 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:36.920454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2c5867ba-9d90-4826-8d93-d46d3faa6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:36.952370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:36.953018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:36.953352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:36.953637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] PATH_INFO: `/v3/projects/f9e048e32c184699b833b7d03b49d3bf` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:36.954998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e048e32c184699b833b7d03b49d3bf)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:36.956867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:36.998828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 919/1842] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:37.006613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.009854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.009854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.028349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:37.029572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:37.038288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 920/1843] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:37.042376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.053018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eadc3-482c-4fb7-b80f-243daf3a00e4 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.058442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.058606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.058781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.060060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.060744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:37.060854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.062459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.083464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 921/1844] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:09:37.084444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.097014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9e9f6-d0d3-4357-9725-c54ca12ef7f8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.102947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e048e32c184699b833b7d03b49d3bf', 'request_id': 'req-49a47e67-633c-4c16-a75b-38ef39435529'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:37.161537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.162181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.162465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.164426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.164426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.165401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.183544 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e048e32c184699b833b7d03b49d3bf', 'request_id': 'req-49a47e67-633c-4c16-a75b-38ef39435529'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:37.194014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e048e32c184699b833b7d03b49d3bf', 'request_id': 'req-49a47e67-633c-4c16-a75b-38ef39435529'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:37.196492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:37.207002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] The token cache is being invalidate because project f9e048e32c184699b833b7d03b49d3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:37.207256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:37.213935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-49a47e67-633c-4c16-a75b-38ef39435529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:37.223755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 924/1845] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:36 2022] DELETE /identity/v3/projects/f9e048e32c184699b833b7d03b49d3bf => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:37.223755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02e1c5-4e58-476d-a452-627280ddef6d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.223755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02e1c5-4e58-476d-a452-627280ddef6d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.223755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02e1c5-4e58-476d-a452-627280ddef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.270333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 922/1846] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:37.270523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7e02e1c5-4e58-476d-a452-627280ddef6d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:37.270523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7e02e1c5-4e58-476d-a452-627280ddef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:37.271985 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.274036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:37.342505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:37.360769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a39f87-bd97-4a03-8c76-406e5147700e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.363225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 925/1847] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:37.366018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.366646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.366931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.367211 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.367848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.369858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.371086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.371518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.371797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.391948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:37.393130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:37.401549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 926/1848] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:37.406177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.417294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e879793-c74a-419e-a3fa-95e954a59cf5 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.425469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.425735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.426901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.427173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.427898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:37.428274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.429889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.433908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 923/1849] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:37.438619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.449730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 927/1850] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:09:37.450097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.456172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d203d9-58fe-47cc-b7e7-fb40a07276bb admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.461372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2cec0-ccb6-4870-ba47-4d723e99c853 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.462993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.463762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.464685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.465679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.466487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.469046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.469218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.469669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.469950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.470224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.470864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.473030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.494313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 924/1851] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:37.498421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.511795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b9c45-e316-45b8-bd96-dfc02939e390 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.516800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.517428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.518472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.518472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.518675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.520295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.533800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 925/1852] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:37.537927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.548440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cd76d-7fa7-4d28-b073-8c5efbb68724 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.554256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1175a-8766-4fd3-a9cf-0fb3ddb0df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.554903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1175a-8766-4fd3-a9cf-0fb3ddb0df94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.555182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1175a-8766-4fd3-a9cf-0fb3ddb0df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.555457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1175a-8766-4fd3-a9cf-0fb3ddb0df94 admin admin] PATH_INFO: `/v3/projects/172c0164aa2c4e53a22784dc5324ca0b/users/eebf57a78ad74565806b1ad6fbe605ff/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.589080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1175a-8766-4fd3-a9cf-0fb3ddb0df94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172c0164aa2c4e53a22784dc5324ca0b, user_id=eebf57a78ad74565806b1ad6fbe605ff, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.595164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 928/1853] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:37.604740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.604885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1175a-8766-4fd3-a9cf-0fb3ddb0df94 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.665897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b69bb-e579-4d3c-8391-ea9885ea76f4 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.672201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.673165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.673651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.674137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.675290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.678121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.716064 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 926/1854] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:37 2022] PUT /identity/v3/projects/172c0164aa2c4e53a22784dc5324ca0b/users/eebf57a78ad74565806b1ad6fbe605ff/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:37.727128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.727249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.727492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.757349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] MFA Rules not processed for user `eebf57a78ad74565806b1ad6fbe605ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:37.758631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:37.801663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 929/1855] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:37.805871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.816903 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418573a0-2637-4cc4-b1ea-49affe15e5ab admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.829711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 927/1856] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:37.849489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:37.852677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:37.888339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:37.956741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fec18c-a418-43b0-98bb-4f583e4764f2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:37.961198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.962493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.962961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.963278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.969830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:37.970379 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:37.970473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:37.970748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:37.971363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.976805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:37.982173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:37.987483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.004018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 930/1857] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:38.020606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 928/1858] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:38.023488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.041675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace124e-eae5-4852-9ef9-ca21507539b3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.053059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3853d-6ced-4f8e-9ce0-d3bf722d9c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.053797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3853d-6ced-4f8e-9ce0-d3bf722d9c79 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.054082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3853d-6ced-4f8e-9ce0-d3bf722d9c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.054427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3853d-6ced-4f8e-9ce0-d3bf722d9c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.058229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3853d-6ced-4f8e-9ce0-d3bf722d9c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.065591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3853d-6ced-4f8e-9ce0-d3bf722d9c79 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.079543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 931/1859] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:38.083403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.102579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dec9b-8fbf-4a23-93d3-ff9d6ad471c5 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.114109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.115219 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.115502 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.116304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] PATH_INFO: `/v3/projects/0fbff0f82f6f4bc18fecec8515b210c7/users/33a92618a2fe41349ea96d750291c42f/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.151833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbff0f82f6f4bc18fecec8515b210c7, user_id=33a92618a2fe41349ea96d750291c42f, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.155637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.232052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:38.240135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 929/1860] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:38 2022] PUT /identity/v3/projects/0fbff0f82f6f4bc18fecec8515b210c7/users/33a92618a2fe41349ea96d750291c42f/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:38.251063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.251349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.251647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.281959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] MFA Rules not processed for user `33a92618a2fe41349ea96d750291c42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:38.283171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:38.368306 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 932/1861] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:38.369231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.493953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c2394-7c52-4cd4-a6f4-5909450cc591 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.499750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.500481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.500764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.501041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.501882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.504485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.619746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 930/1862] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:38 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:38.624124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.644376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd178-4a2a-4d96-840c-7d838ab32d13 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.688055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.689966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:38.710364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.711048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.711473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.714003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.714003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.714880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.777605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1aaba-de75-4ba3-a0c8-6c2efd7d3cc3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.783176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.783818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.784130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.784412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] PATH_INFO: `/v3/users/f6d39546d0504953a4f9d769db9421a8` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.785640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d39546d0504953a4f9d769db9421a8)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.787605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.811196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 933/1863] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:38 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:38.815962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.835082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4862-d071-45cf-a178-5fd666a60a7a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.848509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.849503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.850248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.850669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.851463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.853255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.867563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 934/1864] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:38.876791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.894702 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad1f92-d159-47cd-ab1c-926a37a46ea3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.899542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.900211 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.900487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.900758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.901394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.903006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:38.914219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 935/1865] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:38.918478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:38.929054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32333b-c3a9-4cd7-98bb-1e0f7184e8f7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:38.934377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977a2b6-89a7-4177-bc1d-13f672719464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:38.935035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977a2b6-89a7-4177-bc1d-13f672719464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:38.936182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977a2b6-89a7-4177-bc1d-13f672719464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:38.936182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977a2b6-89a7-4177-bc1d-13f672719464 admin admin] PATH_INFO: `/v3/projects/a1ed7e8179f54b1b8c5fb323e82fb852/users/6a2c19f2ddde47ef8f9cd3431c594726/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:38.955517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977a2b6-89a7-4177-bc1d-13f672719464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ed7e8179f54b1b8c5fb323e82fb852, user_id=6a2c19f2ddde47ef8f9cd3431c594726, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:38.959036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977a2b6-89a7-4177-bc1d-13f672719464 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.045550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 936/1866] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:38 2022] PUT /identity/v3/projects/a1ed7e8179f54b1b8c5fb323e82fb852/users/6a2c19f2ddde47ef8f9cd3431c594726/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:39.055991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.056305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.056579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.058548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d39546d0504953a4f9d769db9421a8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.064072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:39.085804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] MFA Rules not processed for user `6a2c19f2ddde47ef8f9cd3431c594726`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:39.086514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:39.131625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d39546d0504953a4f9d769db9421a8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.132300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d39546d0504953a4f9d769db9421a8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.132971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d39546d0504953a4f9d769db9421a8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.147008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d39546d0504953a4f9d769db9421a8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.151720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 937/1867] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:39.160225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.160371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 931/1868] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:38 2022] DELETE /identity/v3/users/f6d39546d0504953a4f9d769db9421a8 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:39.161742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:39.234275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa7036-a0a1-48c6-b083-7e35e68b3e0c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.304666 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.305220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.305503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.305783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.306420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.308487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.426426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 938/1869] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:39 2022] POST /identity/v3/projects => generated 375 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:39.429455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.436366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:39.439967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c456e-a224-4710-b078-564b06b497e9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.459035 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.460783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:39.494589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:39.501484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.502128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.502423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.502696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.503331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.505310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.531836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99cb3f-7d70-4bdf-9cdb-a0883e60e01e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.536552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.537184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.537462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.537735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] PATH_INFO: `/v3/projects/f9c633053a92460c8fd6858e66679fd3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.538864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c633053a92460c8fd6858e66679fd3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.540836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.610153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 932/1870] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:39 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:39.615004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.625192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1aafb-97f5-4ecc-8919-a56c2b9a8c6c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.630075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.630690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.631098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.631454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.632250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.634010 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.647193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 933/1871] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:39.647351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c633053a92460c8fd6858e66679fd3', 'request_id': 'req-f5e1fc72-3ce0-4c78-b318-82428caa6081'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.648913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.659827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1c3a1-8b7c-40ed-8a2d-a4a6d671e3d0 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.718544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.719215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.719523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.719860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] PATH_INFO: `/v3/projects/b33530e06f0944bf9156c99557d175a1/users/6dc272d760374d4bb6b060ec4e1a9db5/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.727297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c633053a92460c8fd6858e66679fd3', 'request_id': 'req-f5e1fc72-3ce0-4c78-b318-82428caa6081'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.736640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c633053a92460c8fd6858e66679fd3', 'request_id': 'req-f5e1fc72-3ce0-4c78-b318-82428caa6081'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.736881 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33530e06f0944bf9156c99557d175a1, user_id=6dc272d760374d4bb6b060ec4e1a9db5, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.739541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.749065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] The token cache is being invalidate because project f9c633053a92460c8fd6858e66679fd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:39.749361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.750062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f5e1fc72-3ce0-4c78-b318-82428caa6081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:39.751593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 939/1872] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:39 2022] DELETE /identity/v3/projects/f9c633053a92460c8fd6858e66679fd3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:39.756507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7fe4e-e4e7-438d-8990-01d6da5d6f8c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.756789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7fe4e-e4e7-438d-8990-01d6da5d6f8c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.757085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7fe4e-e4e7-438d-8990-01d6da5d6f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.783277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b0e7fe4e-e4e7-438d-8990-01d6da5d6f8c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:39.783277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b0e7fe4e-e4e7-438d-8990-01d6da5d6f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:39.787016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 940/1873] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:39.795514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.795961 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.796316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.817171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:39.818413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:39.826773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 941/1874] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:39.829376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 934/1875] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:39 2022] PUT /identity/v3/projects/b33530e06f0944bf9156c99557d175a1/users/6dc272d760374d4bb6b060ec4e1a9db5/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:39.830985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.832906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.834800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:39.841392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28a50-1b09-4fbe-a947-0e6c033ca866 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.917739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccee917-8961-430b-a29d-ff508e40c10d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.921230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.921947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.922297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.922636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.923384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:09:39.923763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.925652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.934597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.935816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.936239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.936654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.937850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.939645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.941966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 942/1876] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:09:39.947125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.960869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d5528-f365-4fd0-b4ad-a0587d509cfa admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.961445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 935/1877] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:09:39.962520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:39.966952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.967902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.967902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.968524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.969271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.970420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:39.979159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:39.987274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51301213-a438-4f63-bb3f-cc807c0867aa admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:39.993796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d4d54-633b-45ba-a28d-f3d26f6a6df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:39.994484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4d54-633b-45ba-a28d-f3d26f6a6df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:39.994762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4d54-633b-45ba-a28d-f3d26f6a6df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:39.995052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d4d54-633b-45ba-a28d-f3d26f6a6df2 admin admin] PATH_INFO: `/v3/projects/b33530e06f0944bf9156c99557d175a1/users/6dc272d760374d4bb6b060ec4e1a9db5/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:39.997118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d4d54-633b-45ba-a28d-f3d26f6a6df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33530e06f0944bf9156c99557d175a1, user_id=6dc272d760374d4bb6b060ec4e1a9db5, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:39.999709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d4d54-633b-45ba-a28d-f3d26f6a6df2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.013939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 936/1878] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:39 2022] PUT /identity/v3/projects/b33530e06f0944bf9156c99557d175a1/users/6dc272d760374d4bb6b060ec4e1a9db5/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:40.021198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.021495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.021774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.051634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] MFA Rules not processed for user `6dc272d760374d4bb6b060ec4e1a9db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:40.052903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:40.100059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 943/1879] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:09:39 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:09:40.102288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:40.115639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:40.130993 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 937/1880] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:40.141770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:40.143155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:40.173807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632110e-e64a-469e-b9f1-2d5f06891ad0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:40.179947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:40.180906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.181156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.181433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.182598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:40.184160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.205095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e4121-4ce3-487e-b8ef-6d7c0fc8f52b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:40.252967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:40.253581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.253861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.254138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.255930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:40.285881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 944/1881] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:09:40 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:09:40.291719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:40.299755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfe730-9bc7-4f69-9170-e6833956f267 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:40.304724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:40.305327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.305613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.305879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.306517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:40.308304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.313927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:40.318691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:40.331185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.331322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 945/1882] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:40.332041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 938/1883] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:40.334076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:40.346756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e105d42-707a-4fd4-9794-84724e19ddde admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:40.351745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd8b1-55fa-4c52-aef8-f1c9c8347373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:40.352514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd8b1-55fa-4c52-aef8-f1c9c8347373 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.352796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd8b1-55fa-4c52-aef8-f1c9c8347373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.353091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd8b1-55fa-4c52-aef8-f1c9c8347373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.353770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbd8b1-55fa-4c52-aef8-f1c9c8347373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:40.359165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbd8b1-55fa-4c52-aef8-f1c9c8347373 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.372609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 946/1884] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:09:40.377163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:40.389069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24a7c3-cb79-4900-8196-ea043764b00c service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:40.393677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:40.394506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.394659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.394892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] PATH_INFO: `/v3/projects/2488f5601066432a8652433df5702119/users/1e8d2e4a8a334eea973280b3819219e3/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.410654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2488f5601066432a8652433df5702119, user_id=1e8d2e4a8a334eea973280b3819219e3, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:40.413343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.506144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 939/1885] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:09:40 2022] PUT /identity/v3/projects/2488f5601066432a8652433df5702119/users/1e8d2e4a8a334eea973280b3819219e3/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:40.511818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.512778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.513146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.533105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:40.534477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:40.600485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 947/1886] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:40.610167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:40.611995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:40.671779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ece8-edc1-4659-90e3-d026e97e69d3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:40.675587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94de01d-dd67-44f5-b33f-ecbd11b845c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:40.676296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de01d-dd67-44f5-b33f-ecbd11b845c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:40.676640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de01d-dd67-44f5-b33f-ecbd11b845c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:40.676979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94de01d-dd67-44f5-b33f-ecbd11b845c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:40.686526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de01d-dd67-44f5-b33f-ecbd11b845c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:40.690989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94de01d-dd67-44f5-b33f-ecbd11b845c9 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:40.697776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 940/1887] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:44.757773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:44.826909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eac712-630c-4aa7-8acc-728f806c8c50 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:44.831031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:44.831762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:44.832153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:44.832439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:44.834495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:44.882991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:44.946290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:44.948518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869187-f83b-4e96-8df3-54305ba44131 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:44.961510 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 948/1888] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:46.098833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:46.099470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:46.099470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:46.131621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] MFA Rules not processed for user `1e8d2e4a8a334eea973280b3819219e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:46.132927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:46.220073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 941/1889] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:46.229439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8295131-2b76-4bd9-b446-f1b8dfcda70d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:46.229990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8295131-2b76-4bd9-b446-f1b8dfcda70d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:46.229990 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8295131-2b76-4bd9-b446-f1b8dfcda70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:46.251033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c8295131-2b76-4bd9-b446-f1b8dfcda70d None None] MFA Rules not processed for user `1e8d2e4a8a334eea973280b3819219e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:46.252305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c8295131-2b76-4bd9-b446-f1b8dfcda70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:46.259507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 949/1890] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:46.273096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:46.274524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:46.337747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:46.364707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8bb7-6b4f-48e4-84e2-29c28ac00d09 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:46.370731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:46.372011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:46.372331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:46.372605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:46.381557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:46.384389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:46.398649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 942/1891] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:09:47.099972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:47.100243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:47.100517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:47.119099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] MFA Rules not processed for user `1e8d2e4a8a334eea973280b3819219e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:47.120336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:47.128562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 950/1892] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:48.954004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:48.963834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd02ed-da55-4b0a-a840-dc5864d3664a service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:48.967757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c7c5c-544f-4717-be79-a54901fed409 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:48.968489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c7c5c-544f-4717-be79-a54901fed409 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:48.968944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c7c5c-544f-4717-be79-a54901fed409 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:48.969370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c7c5c-544f-4717-be79-a54901fed409 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:49.039172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c7c5c-544f-4717-be79-a54901fed409 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:49.040652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c7c5c-544f-4717-be79-a54901fed409 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:49.052233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 943/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:57.114175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=17 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:57.630580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:57.648964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e7ef3-5180-47df-9d0c-f9bd1b560317 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:57.656876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:57.657813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:57.658715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:57.659008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] PATH_INFO: `/v3/users/33a92618a2fe41349ea96d750291c42f` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:57.660988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a92618a2fe41349ea96d750291c42f)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:57.666822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:57.901813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a92618a2fe41349ea96d750291c42f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:57.902506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a92618a2fe41349ea96d750291c42f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:57.988506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a92618a2fe41349ea96d750291c42f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.001824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a92618a2fe41349ea96d750291c42f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.002518 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a92618a2fe41349ea96d750291c42f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.014833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 951/1894] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:57 2022] DELETE /identity/v3/users/33a92618a2fe41349ea96d750291c42f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:58.030486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:58.030942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:58.031031 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:58.051853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:58.053098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:58.115324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 944/1895] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:09:58.126907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:58.128403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:58.186677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fda28-8ae2-44d0-821f-f267757e5ccc admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:58.190371 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:58.191246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:58.191526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:58.191800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:58.201580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:58.202995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:58.212334 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 952/1896] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:58.668189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:58.669642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:58.681734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae6b4-72ff-4846-a79d-3b3e6408454a None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:58.686952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:58.693918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:58.693918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:58.693918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] PATH_INFO: `/v3/projects/0fbff0f82f6f4bc18fecec8515b210c7` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:58.693918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fbff0f82f6f4bc18fecec8515b210c7)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:58.693918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:58.856693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fbff0f82f6f4bc18fecec8515b210c7', 'request_id': 'req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.862540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fbff0f82f6f4bc18fecec8515b210c7', 'request_id': 'req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.869606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fbff0f82f6f4bc18fecec8515b210c7', 'request_id': 'req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.881543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:58.958767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] The token cache is being invalidate because project 0fbff0f82f6f4bc18fecec8515b210c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:09:58.959062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.959694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:58.961218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 945/1897] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:58 2022] DELETE /identity/v3/projects/0fbff0f82f6f4bc18fecec8515b210c7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:58.966670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:58.976167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5e39d-7d7b-4f6e-a204-4810b818b913 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:59.032153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:59.032783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:59.033068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:59.033344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] PATH_INFO: `/v3/users/6a2c19f2ddde47ef8f9cd3431c594726` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:59.034462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2c19f2ddde47ef8f9cd3431c594726)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:59.036463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:59.055941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:09:59.237091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:59.238550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:59.251804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe83dcd-34bb-4894-9cbb-c6cca28c1e65 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:59.256652 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:59.257278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:59.257554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:59.257823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] PATH_INFO: `/v3/users/0c32c28566da4cccaf6c0b123fd9fc2e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:59.258962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c32c28566da4cccaf6c0b123fd9fc2e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:59.259572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2c19f2ddde47ef8f9cd3431c594726'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.260446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2c19f2ddde47ef8f9cd3431c594726'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.260891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:59.353803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2c19f2ddde47ef8f9cd3431c594726'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.365947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2c19f2ddde47ef8f9cd3431c594726'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.366596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dfa09c31-dec4-4929-906c-3b4e6614e38b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2c19f2ddde47ef8f9cd3431c594726'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.378669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 953/1898] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:58 2022] DELETE /identity/v3/users/6a2c19f2ddde47ef8f9cd3431c594726 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:09:59.421738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c32c28566da4cccaf6c0b123fd9fc2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.507115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c32c28566da4cccaf6c0b123fd9fc2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.507754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c32c28566da4cccaf6c0b123fd9fc2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.508379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c32c28566da4cccaf6c0b123fd9fc2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.521173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c32c28566da4cccaf6c0b123fd9fc2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:09:59.532980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 946/1899] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:09:59 2022] DELETE /identity/v3/users/0c32c28566da4cccaf6c0b123fd9fc2e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:09:59.537400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:59.537683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:59.537954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:59.558849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:09:59.559671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:59.642901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 954/1900] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:09:59.658540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:59.660267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:59.735126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f05aa-ebff-4249-a953-4b806e3b9335 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:59.740287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:59.740902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:59.741244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:59.741517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:59.751928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:59.753712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:09:59.768444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 947/1901] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:09:59.903532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:09:59.904998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:09:59.917406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca3e01-f516-4173-94e5-5c612a0b40e3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:09:59.923048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:09:59.923836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:09:59.924154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:09:59.924731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] PATH_INFO: `/v3/projects/a1ed7e8179f54b1b8c5fb323e82fb852` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:09:59.925598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ed7e8179f54b1b8c5fb323e82fb852)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:09:59.927497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:00.046020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ed7e8179f54b1b8c5fb323e82fb852', 'request_id': 'req-973001b6-ee5b-48df-99a0-f2a184e221cf'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.131070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ed7e8179f54b1b8c5fb323e82fb852', 'request_id': 'req-973001b6-ee5b-48df-99a0-f2a184e221cf'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.137450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ed7e8179f54b1b8c5fb323e82fb852', 'request_id': 'req-973001b6-ee5b-48df-99a0-f2a184e221cf'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.149643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] The token cache is being invalidate because project a1ed7e8179f54b1b8c5fb323e82fb852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:00.149935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.150588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.153440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 955/1902] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:09:59 2022] DELETE /identity/v3/projects/a1ed7e8179f54b1b8c5fb323e82fb852 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:00.158018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:00.167442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea8da-83e7-4ee5-9487-1ff056041c65 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:00.223724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:00.224431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:00.224703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:00.224980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] PATH_INFO: `/v3/users/6dc272d760374d4bb6b060ec4e1a9db5` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:00.226103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc272d760374d4bb6b060ec4e1a9db5)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:00.228052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:00.241049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:00.257422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:00.258963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:00.282560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973001b6-ee5b-48df-99a0-f2a184e221cf admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:00.289092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:00.289741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:00.290023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:00.290309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] PATH_INFO: `/v3/projects/741b2e3c05224086b569304d6ae3682a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:00.291439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741b2e3c05224086b569304d6ae3682a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:00.293433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:00.306483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:00.401583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc272d760374d4bb6b060ec4e1a9db5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.406644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741b2e3c05224086b569304d6ae3682a', 'request_id': 'req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.496953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc272d760374d4bb6b060ec4e1a9db5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.497593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc272d760374d4bb6b060ec4e1a9db5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.498306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc272d760374d4bb6b060ec4e1a9db5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.510760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc272d760374d4bb6b060ec4e1a9db5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.523435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 948/1903] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:00 2022] DELETE /identity/v3/users/6dc272d760374d4bb6b060ec4e1a9db5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:00.571354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741b2e3c05224086b569304d6ae3682a', 'request_id': 'req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.578894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741b2e3c05224086b569304d6ae3682a', 'request_id': 'req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.594710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] The token cache is being invalidate because project 741b2e3c05224086b569304d6ae3682a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:00.594999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.595684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:00.597223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 956/1904] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:00 2022] DELETE /identity/v3/projects/741b2e3c05224086b569304d6ae3682a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:00.601637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:00.603971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:00.668850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837b48-00d2-46d2-aec7-c4a00d416747 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:00.673994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:00.674617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:00.674893 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:00.675164 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] PATH_INFO: `/v3/users/e2628deba0f84b77b3f0bb2bf3634f03` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:00.676303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2628deba0f84b77b3f0bb2bf3634f03)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:00.678376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:00.886329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2628deba0f84b77b3f0bb2bf3634f03'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.034413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2628deba0f84b77b3f0bb2bf3634f03'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.035254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2628deba0f84b77b3f0bb2bf3634f03'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.036072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2628deba0f84b77b3f0bb2bf3634f03'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.049838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2628deba0f84b77b3f0bb2bf3634f03'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.066270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 949/1905] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:00 2022] DELETE /identity/v3/users/e2628deba0f84b77b3f0bb2bf3634f03 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:01.102543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:01.104169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.183562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ef0e3-7fd5-410d-9098-ae5b9eb5f834 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:01.241173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:01.241839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.242116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.242456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] PATH_INFO: `/v3/projects/b33530e06f0944bf9156c99557d175a1` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.243611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33530e06f0944bf9156c99557d175a1)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:01.245534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:01.360420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33530e06f0944bf9156c99557d175a1', 'request_id': 'req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.362080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:01.473747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33530e06f0944bf9156c99557d175a1', 'request_id': 'req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.479916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33530e06f0944bf9156c99557d175a1', 'request_id': 'req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.480903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:01.482462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.496638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb709ea-baec-42b7-ae05-6ee8019ef61b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:01.501632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] The token cache is being invalidate because project b33530e06f0944bf9156c99557d175a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:01.502149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.502914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-af8c7cbd-48dc-48a5-87ec-dd0c18e8147a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.504058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:01.505203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 957/1906] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:01 2022] DELETE /identity/v3/projects/b33530e06f0944bf9156c99557d175a1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:01.510701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.511040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.511324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.533391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:01.534642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.572331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:01.573168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.573591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.573876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] PATH_INFO: `/v3/projects/06939da2aa95417b9c954b72202263d2` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.575063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=06939da2aa95417b9c954b72202263d2)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:01.577069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:01.595708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 958/1907] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:01.596672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:01.597987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.610157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a14d-3918-45ad-8a07-1607fb9791b8 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:01.614777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:01.615398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.615677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.615948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] PATH_INFO: `/v3/users/7fb0eb26fe604c02a5db5a114202c127` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.617108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb0eb26fe604c02a5db5a114202c127)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:01.619000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:01.684508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06939da2aa95417b9c954b72202263d2', 'request_id': 'req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.689868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06939da2aa95417b9c954b72202263d2', 'request_id': 'req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.696339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06939da2aa95417b9c954b72202263d2', 'request_id': 'req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.787367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] The token cache is being invalidate because project 06939da2aa95417b9c954b72202263d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:01.797895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.797895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-04e7b91f-26dd-4783-8b4c-2bb10d56d02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.797895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 950/1908] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:01 2022] DELETE /identity/v3/projects/06939da2aa95417b9c954b72202263d2 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:01.797895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236cdd3-f93c-45c6-b8b0-3dbfa065895c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.797895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236cdd3-f93c-45c6-b8b0-3dbfa065895c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.797895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236cdd3-f93c-45c6-b8b0-3dbfa065895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.812296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6236cdd3-f93c-45c6-b8b0-3dbfa065895c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:01.813833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6236cdd3-f93c-45c6-b8b0-3dbfa065895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.875832 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb0eb26fe604c02a5db5a114202c127'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.876654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb0eb26fe604c02a5db5a114202c127'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.882389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 951/1909] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:01.883736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034904e-7895-483e-b129-94d84c869b7f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.884459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034904e-7895-483e-b129-94d84c869b7f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.884459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034904e-7895-483e-b129-94d84c869b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.903618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8034904e-7895-483e-b129-94d84c869b7f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:01.906968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8034904e-7895-483e-b129-94d84c869b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.913406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 952/1910] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:01.914195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034904e-7895-483e-b129-94d84c869b7f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:01.915375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8034904e-7895-483e-b129-94d84c869b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.927668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034904e-7895-483e-b129-94d84c869b7f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:01.932557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:01.933181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.933498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.933767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.934430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:01.934730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:01.936367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d488f-5557-4e18-8944-f41cfe9b4a68 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:01.951783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 953/1911] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:10:01.951783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.951783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.951783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.969734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:01.971862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:01.978753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 954/1912] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:01.979546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:01.983430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb0eb26fe604c02a5db5a114202c127'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:01.992613 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b67e0-f47f-45cc-81ef-e73683b886d5 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:01.997441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:01.998089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:01.998446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:01.998901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:01.999453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.001171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb0eb26fe604c02a5db5a114202c127'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:02.001862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.002170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb0eb26fe604c02a5db5a114202c127'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:02.016945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 959/1913] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:01 2022] DELETE /identity/v3/users/7fb0eb26fe604c02a5db5a114202c127 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:02.020809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.050500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:02.072170 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:02.083523 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c495e4-91db-4971-a13b-64bf8a3f2199 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.088166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.088772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.089052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.089321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.089958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:02.090256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.091862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c379d94-5d5c-448e-9a3b-90050416715a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.107375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 960/1914] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:10:02.107375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.107375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.109633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.114561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 955/1915] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:01 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:02.115542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.129141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:02.130364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:02.187491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94de4-60b0-4584-9bfe-a44ee0b5d473 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.193182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.193818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.194076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.194351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.195043 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.196983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.226069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 961/1916] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:02.231488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.239667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0faea-9133-4009-a255-c2d05a497796 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.244519 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.245232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.245572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.245851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.246555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.248609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.283059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 956/1917] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:02 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:02.283717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.285475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:02.332086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 962/1918] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:02 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:02.332987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.347139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d52e-52df-46d5-b39f-9eaab5dd1182 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.356519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c56b6-85ed-44c8-8b33-ccc86ab1a24a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.360306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.361089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.361795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.362252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.432590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.436393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.436393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.436393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.436393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.436987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.460761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.462187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.471899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 957/1919] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:02.472879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.482042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b30a98-05a1-4a75-bd99-f25f5fdf647e service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.487339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.487943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.488256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.488529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.489176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.490925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.503385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 958/1920] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:02.507405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.519704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3ab4-df92-4f4e-a95e-cfd954b95ed8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.524612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.525218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.525508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.525784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.526460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.528226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.541027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 959/1921] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:02.541413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 963/1922] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:02 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:02.546327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.548133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.555212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:02.559538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a3bd1-f2f1-4a91-abee-856748d52189 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.565094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779d0af-4596-464d-af6e-1c3906958927 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.565358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae3e9a-df1f-4ae3-8787-1ea28a022e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.566307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae3e9a-df1f-4ae3-8787-1ea28a022e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.566564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae3e9a-df1f-4ae3-8787-1ea28a022e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.566841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae3e9a-df1f-4ae3-8787-1ea28a022e20 admin admin] PATH_INFO: `/v3/projects/2fb4a9a2be5947f1abc23fae594eaae9/users/5ebece46244e4f8facd33f3b9dadcec4/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.572432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.573312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.573312 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.573551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.574146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.575878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.585216 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae3e9a-df1f-4ae3-8787-1ea28a022e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb4a9a2be5947f1abc23fae594eaae9, user_id=5ebece46244e4f8facd33f3b9dadcec4, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.586825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 964/1923] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:02.588144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae3e9a-df1f-4ae3-8787-1ea28a022e20 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.590880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.601907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad4fa1-2a18-4dc8-8092-7c846faa967b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.606588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.607178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.607454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.607798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.608454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.610033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.620960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 965/1924] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:02.625125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.636223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9c1a-e1ad-4e9f-b4d7-fe1b1ad8ea94 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.636383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 960/1925] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:02 2022] PUT /identity/v3/projects/2fb4a9a2be5947f1abc23fae594eaae9/users/5ebece46244e4f8facd33f3b9dadcec4/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:02.648098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.648392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.648659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.673045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:02.691470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] MFA Rules not processed for user `5ebece46244e4f8facd33f3b9dadcec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:02.692931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:02.706543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffbe32-6aee-43ce-a525-d09866c63d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.708326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffbe32-6aee-43ce-a525-d09866c63d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.708574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffbe32-6aee-43ce-a525-d09866c63d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.708940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffbe32-6aee-43ce-a525-d09866c63d85 admin admin] PATH_INFO: `/v3/projects/fc092aa40cc74450b5e652b85b616424/users/cf3e419c53e94c05a98dcc874179526b/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.724705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffbe32-6aee-43ce-a525-d09866c63d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc092aa40cc74450b5e652b85b616424, user_id=cf3e419c53e94c05a98dcc874179526b, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:02.728924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffbe32-6aee-43ce-a525-d09866c63d85 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:02.761719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 961/1926] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:02.774794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.776430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:02.810002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 966/1927] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:02 2022] PUT /identity/v3/projects/fc092aa40cc74450b5e652b85b616424/users/cf3e419c53e94c05a98dcc874179526b/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:02.822950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.823566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.824005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.850961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4899365-4833-4a1f-9759-341f0a38a652 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:02.858693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] MFA Rules not processed for user `cf3e419c53e94c05a98dcc874179526b`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:02.860453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:02.912538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:02.918770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:02.918770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:02.918770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:02.951070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 967/1928] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:02.952000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:02.953722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:02.973210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:03.001022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.002470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.013684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 962/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:03.014567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.084208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71cd1f-d8df-4f7c-90eb-7ad8581db0f8 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.090019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.092154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.092431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.092708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] PATH_INFO: `/v3/projects/4abcd297b89c4351a268c6c4e972de80` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.094524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4abcd297b89c4351a268c6c4e972de80)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.096507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.099023 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262553f-18b9-4300-aabb-9b9c867091f0 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.105434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.106407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.107308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.108509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.109685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.112340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.201051 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4abcd297b89c4351a268c6c4e972de80', 'request_id': 'req-ac95ab89-1788-49a3-b050-66c60dd6512c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:03.202826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:03.241892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 963/1930] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:03.245710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.257192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4abcd297b89c4351a268c6c4e972de80', 'request_id': 'req-ac95ab89-1788-49a3-b050-66c60dd6512c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:03.266434 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4abcd297b89c4351a268c6c4e972de80', 'request_id': 'req-ac95ab89-1788-49a3-b050-66c60dd6512c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:03.301749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] The token cache is being invalidate because project 4abcd297b89c4351a268c6c4e972de80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:03.302621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:03.303384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ac95ab89-1788-49a3-b050-66c60dd6512c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:03.305454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 968/1931] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:02 2022] DELETE /identity/v3/projects/4abcd297b89c4351a268c6c4e972de80 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:03.311477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b63798-4024-4e18-be7d-9fbfed115d5a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.315042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b63798-4024-4e18-be7d-9fbfed115d5a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.315042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b63798-4024-4e18-be7d-9fbfed115d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.333808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ae52-effd-4dac-9c46-5024f448d3df admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.338160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e0b63798-4024-4e18-be7d-9fbfed115d5a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:03.338621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e0b63798-4024-4e18-be7d-9fbfed115d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:03.340942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.342674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.343413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.343998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.344791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.346786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.366078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 969/1932] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:03.367678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.368066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.368314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.415686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:03.416973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:03.432706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 970/1933] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:03.436684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.447025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469e3cd-4e7c-4c7c-b7c4-20cca30afbd3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.460962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 964/1934] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:10:03.461071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.461071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.461071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.461755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.463469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:03.464747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.482196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:03.485669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.486381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.500531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ffa57-1ef0-464e-afa0-af4b933ad1d5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.506536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.515346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.515346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.515346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.515346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.515346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.524082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 971/1935] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:10:03.525268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 965/1936] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:03.528426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.535379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.540107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76de48-b084-46c3-856e-ec8d011a6194 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.546347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.547251 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.547349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.547685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.548361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.550852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.555308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89167b3-424e-439f-ad35-e190e54bf907 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.562907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.563612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.563971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.564749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.566911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.568542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.583949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 966/1937] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:03.584748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.608555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85f237-b8c7-4787-b649-adb4fc34a83e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.619746 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03906e-0e97-4f0e-87f6-afd02d4e79fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.627723 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03906e-0e97-4f0e-87f6-afd02d4e79fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.627723 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03906e-0e97-4f0e-87f6-afd02d4e79fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.627723 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03906e-0e97-4f0e-87f6-afd02d4e79fd admin admin] PATH_INFO: `/v3/projects/1044a24fb8f4407895c6fa6d4d1f5aab/users/a7ef031e0ad04e278090bb2b5e4868da/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.646511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03906e-0e97-4f0e-87f6-afd02d4e79fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1044a24fb8f4407895c6fa6d4d1f5aab, user_id=a7ef031e0ad04e278090bb2b5e4868da, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.651735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03906e-0e97-4f0e-87f6-afd02d4e79fd admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.660305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 972/1938] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:10:03.662021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.748643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977512ac-9155-43d3-89c0-0c5250bd30de admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.755472 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.756233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.756522 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.756798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.759244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.759551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.761149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 967/1939] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:03 2022] PUT /identity/v3/projects/1044a24fb8f4407895c6fa6d4d1f5aab/users/a7ef031e0ad04e278090bb2b5e4868da/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:03.772313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.772600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.772879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.797150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:03.818813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] MFA Rules not processed for user `a7ef031e0ad04e278090bb2b5e4868da`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:03.820094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:03.898426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 968/1940] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:03.902908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.954620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 973/1941] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:03.956330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:03.967482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59cd447-c0d1-4905-8e62-a5781ce2d80d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.972541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.972710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324da54-a8d5-47fd-947e-98496dafee67 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:03.973190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.973426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.973710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.974335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.975916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.977660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345c4f9-c7ff-45b3-b984-1d26c177ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:03.978268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c4f9-c7ff-45b3-b984-1d26c177ff28 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:03.978679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c4f9-c7ff-45b3-b984-1d26c177ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:03.978961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c4f9-c7ff-45b3-b984-1d26c177ff28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:03.979600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345c4f9-c7ff-45b3-b984-1d26c177ff28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:03.981584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345c4f9-c7ff-45b3-b984-1d26c177ff28 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:03.986261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 974/1942] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:03.990231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.002003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbb0a7-efcb-4371-81e5-5d6357943edc admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.007233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.007834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.008143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.008416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.009046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.010659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.022701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 975/1943] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:04.027000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.036964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df370-a3a4-49bb-8d99-b67b5213d727 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.041703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.042344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.042619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.042886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] PATH_INFO: `/v3/projects/bb22ce3005aa43b09352e92e64ae2122/users/99aaa7ac95a545178fb3ee8b7877b74f/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.058388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb22ce3005aa43b09352e92e64ae2122, user_id=99aaa7ac95a545178fb3ee8b7877b74f, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.061632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.137443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 969/1944] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:03 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:10:04.142579 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 976/1945] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:04 2022] PUT /identity/v3/projects/bb22ce3005aa43b09352e92e64ae2122/users/99aaa7ac95a545178fb3ee8b7877b74f/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:04.147149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.153165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.155890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.156206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.201236 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] MFA Rules not processed for user `99aaa7ac95a545178fb3ee8b7877b74f`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:04.203461 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:04.230864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6bb3-6aec-4af4-895b-f35173f97051 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.268607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 970/1946] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:04.281250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.282731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:04.287772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.288422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.288705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.288979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.289613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.291500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.339237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e348fc7-b169-49f2-961a-591be6e80ec3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.342953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.343556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.343834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.344153 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.355367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.355367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.355576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 977/1947] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:04 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:04.358914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.363824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 971/1948] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:10:04.368959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ca0f1-9496-466a-ac1a-3d596bc87f29 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.374450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.375046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.375321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.375588 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.376255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.378422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.388624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 978/1949] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:04.397220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.409304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24df476-475d-4523-8b5b-75264bd827e2 service glance] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.415161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.415864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.416249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.416523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] PATH_INFO: `/v3/projects/b375260800574b6fbe19b3409f92a5bb/users/daad6697459643da86a056a89dd124ca/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.435179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b375260800574b6fbe19b3409f92a5bb, user_id=daad6697459643da86a056a89dd124ca, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.438481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.440483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:INSERT=6,keystone:DELETE=3,keystone:SELECT=91 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:04.547500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 972/1950] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:04 2022] PUT /identity/v3/projects/b375260800574b6fbe19b3409f92a5bb/users/daad6697459643da86a056a89dd124ca/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:04.551393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.562208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785fcc-8009-4094-a500-84d4de0cabb8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.621722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85902a7a-fd1c-473b-810b-99028cb69ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.622256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85902a7a-fd1c-473b-810b-99028cb69ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.622535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85902a7a-fd1c-473b-810b-99028cb69ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.622806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85902a7a-fd1c-473b-810b-99028cb69ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.624132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85902a7a-fd1c-473b-810b-99028cb69ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.625734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85902a7a-fd1c-473b-810b-99028cb69ac1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.636182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 979/1951] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:04.640756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.651050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d03276-aa30-404e-89c1-c48b15880df2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.656903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.657572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.658163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.658163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] PATH_INFO: `/v3/projects/b375260800574b6fbe19b3409f92a5bb/users/daad6697459643da86a056a89dd124ca/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.660141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b375260800574b6fbe19b3409f92a5bb, user_id=daad6697459643da86a056a89dd124ca, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.663015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.671709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:04.676477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 973/1952] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:04 2022] PUT /identity/v3/projects/b375260800574b6fbe19b3409f92a5bb/users/daad6697459643da86a056a89dd124ca/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:04.687978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.688298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.688577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.720826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] MFA Rules not processed for user `daad6697459643da86a056a89dd124ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:04.722274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:04.808658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 980/1953] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:04.813249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:04.816268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:04.895544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8afd89-6b55-4da3-a370-6e2aafb8eaa7 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:04.900707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:04.901424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:04.901674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:04.901947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:04.913209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:04.915285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:04.928588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 974/1954] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:05.731573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:06.473771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:06.485407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14719165-dba5-45d7-b536-0f87b845a442 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:06.489119 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:06.489817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:06.490126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:06.490429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:06.492201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:06.554772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:06.556163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74931b-c8e5-4ef9-a9ee-255bc0317206 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:06.565132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 981/1955] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:06.724160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:06.725692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:06.795856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289102c-cde8-46e3-a8e6-03ba841b856d service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:06.800810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:06.801438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:06.801720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:06.801994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] PATH_INFO: `/v3/users/5ebece46244e4f8facd33f3b9dadcec4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:06.803165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ebece46244e4f8facd33f3b9dadcec4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:06.805166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:06.830069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:07.020491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ebece46244e4f8facd33f3b9dadcec4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:07.114142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ebece46244e4f8facd33f3b9dadcec4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:07.114832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ebece46244e4f8facd33f3b9dadcec4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:07.115428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ebece46244e4f8facd33f3b9dadcec4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:07.128898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ebece46244e4f8facd33f3b9dadcec4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:07.141305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 975/1956] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:06 2022] DELETE /identity/v3/users/5ebece46244e4f8facd33f3b9dadcec4 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:07.146101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:07.146585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:07.146701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:07.165988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:07.167190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:07.244649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 982/1957] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:07.261028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:07.262788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:07.361607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41605c6-8220-4ab2-8e75-448c2fcd752e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:07.365607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:07.366294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:07.366593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:07.366901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:07.379287 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:07.384374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:07.393448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 976/1958] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:10:07.796267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:07.798436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:07.819927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f373c-645b-40f5-9ebe-5a2671991cdc None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:07.819927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:07.819927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:07.819927 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:07.820486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] PATH_INFO: `/v3/projects/2fb4a9a2be5947f1abc23fae594eaae9` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:07.824352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb4a9a2be5947f1abc23fae594eaae9)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:07.826369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:07.921377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:07.922818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:07.935888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b837c-e6b8-4cf0-b330-7769aa869fa1 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:07.940532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:07.941402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:07.941681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:07.941953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] PATH_INFO: `/v3/users/eebf57a78ad74565806b1ad6fbe605ff` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:07.943080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eebf57a78ad74565806b1ad6fbe605ff)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:07.945014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:07.978517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb4a9a2be5947f1abc23fae594eaae9', 'request_id': 'req-01340c82-977c-4a7a-8374-1267ebc89376'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.068699 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb4a9a2be5947f1abc23fae594eaae9', 'request_id': 'req-01340c82-977c-4a7a-8374-1267ebc89376'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.076019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb4a9a2be5947f1abc23fae594eaae9', 'request_id': 'req-01340c82-977c-4a7a-8374-1267ebc89376'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.088495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] The token cache is being invalidate because project 2fb4a9a2be5947f1abc23fae594eaae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:08.088783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.089660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-01340c82-977c-4a7a-8374-1267ebc89376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.091179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 983/1959] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:07 2022] DELETE /identity/v3/projects/2fb4a9a2be5947f1abc23fae594eaae9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:08.096508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be09c-14f3-4d07-88eb-4359b049e0f0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.096808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be09c-14f3-4d07-88eb-4359b049e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.097078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be09c-14f3-4d07-88eb-4359b049e0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.105021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:08.118244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7d0be09c-14f3-4d07-88eb-4359b049e0f0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:08.120164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7d0be09c-14f3-4d07-88eb-4359b049e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:08.177838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eebf57a78ad74565806b1ad6fbe605ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.193521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 984/1960] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:08.195669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.195944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.196245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.220088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:08.221532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:08.229441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 985/1961] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:08.233675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.242933 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e1c3f-490a-4521-b759-5b676e5139a5 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.247555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.248319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.248586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.248857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.249494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:08.249799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.251375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.262731 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 986/1962] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:10:08.268194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.273267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eebf57a78ad74565806b1ad6fbe605ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.274078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eebf57a78ad74565806b1ad6fbe605ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.274732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eebf57a78ad74565806b1ad6fbe605ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.278790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cff1d-683f-4f39-8283-d75e3ca53fe8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.284085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.284701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.284984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.285370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.286014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.287908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.288656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-a49da347-e934-4ff4-ac46-c864b19c1d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eebf57a78ad74565806b1ad6fbe605ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:08.301528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 977/1963] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:07 2022] DELETE /identity/v3/users/eebf57a78ad74565806b1ad6fbe605ff => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:08.307300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.307300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.307300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.324861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:08.326132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:08.336094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:08.388623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 978/1964] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:08.398803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.400379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:08.416455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 987/1965] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:08 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:10:08.417567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.420262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:08.484490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b2e2a-2631-49a3-b6cc-1f19498c61d9 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.499556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7235e1-2177-4554-893b-318a0f38cfb1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.505063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.505827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.506370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.506487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.507322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.509766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.575727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.577061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.577728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.577907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.601172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 988/1966] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:08 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:08.604621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.606214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.606501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.620439 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 979/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:10:08.626044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96ecb-17ef-4525-93a6-6245ff2c5e37 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.631146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.631833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.632151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.632427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.633073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.634817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.646553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 989/1968] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:08.650989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.662286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb244-cb56-4c6c-a5bf-b1ef924941bf service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.668317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af92889-9f80-47a5-a018-eeda04f18e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.669773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af92889-9f80-47a5-a018-eeda04f18e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.669773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af92889-9f80-47a5-a018-eeda04f18e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.669773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af92889-9f80-47a5-a018-eeda04f18e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.670567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af92889-9f80-47a5-a018-eeda04f18e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.672084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af92889-9f80-47a5-a018-eeda04f18e57 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.687922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 980/1969] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:08.689859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.705573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d7a40-9a4f-4e8c-a2ee-f45506871834 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:08.714528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:08.715444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.715795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.716200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] PATH_INFO: `/v3/projects/9ea1d9bac2d548fea62e9d540a5fbcc0/users/211b7023c0c642f6b85b915658838a3a/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.746943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea1d9bac2d548fea62e9d540a5fbcc0, user_id=211b7023c0c642f6b85b915658838a3a, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:08.751147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:08.840494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 990/1970] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:08 2022] PUT /identity/v3/projects/9ea1d9bac2d548fea62e9d540a5fbcc0/users/211b7023c0c642f6b85b915658838a3a/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:08.851830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:08.852148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:08.852422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:08.889574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] MFA Rules not processed for user `211b7023c0c642f6b85b915658838a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:08.890806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:08.960631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 981/1971] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:08.976177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:08.979066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:09.000562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:09.011477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:09.039942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8647655-fb5a-452a-a8c3-4f6f0829b0b7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:09.043572 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c185-3aeb-4a06-b31d-a134da73ad16 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:09.044240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c185-3aeb-4a06-b31d-a134da73ad16 service glance] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.044521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c185-3aeb-4a06-b31d-a134da73ad16 service glance] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.045938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c185-3aeb-4a06-b31d-a134da73ad16 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.053420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:09.055954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:09.057901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5c185-3aeb-4a06-b31d-a134da73ad16 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:09.060631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5c185-3aeb-4a06-b31d-a134da73ad16 service glance] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:09.079696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 991/1972] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:09.158058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14317312-6f9d-45bc-ba22-939558051dd5 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:09.163025 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:09.163556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.163835 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.165743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] PATH_INFO: `/v3/projects/172c0164aa2c4e53a22784dc5324ca0b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.167272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] RBAC: Authorizing `identity:delete_project(project_id=172c0164aa2c4e53a22784dc5324ca0b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:09.169492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:09.275482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172c0164aa2c4e53a22784dc5324ca0b', 'request_id': 'req-62938b72-fe43-48fc-a866-35ac1646386f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:09.282151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172c0164aa2c4e53a22784dc5324ca0b', 'request_id': 'req-62938b72-fe43-48fc-a866-35ac1646386f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:09.299052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172c0164aa2c4e53a22784dc5324ca0b', 'request_id': 'req-62938b72-fe43-48fc-a866-35ac1646386f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:09.359619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] The token cache is being invalidate because project 172c0164aa2c4e53a22784dc5324ca0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:09.360395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:09.361230 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-62938b72-fe43-48fc-a866-35ac1646386f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:09.363088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 982/1973] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:09 2022] DELETE /identity/v3/projects/172c0164aa2c4e53a22784dc5324ca0b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:09.368227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.369135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.369413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.397115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:09.398350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:09.481202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 992/1974] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:09.492946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.493223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.493575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.514274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:09.515918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:09.525857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 983/1975] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:09.530491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:09.540104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae888e4-1ff5-470e-928f-4c50a938a2ff None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:09.544692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:09.545941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.546247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.546477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.547120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:09.547767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:09.549126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:09.561497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 993/1976] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:10:09.578664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:09.578664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7f7c3-a6d0-4289-92bb-8ec5298cf1a8 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:09.594386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:09.595060 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.595373 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.595712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.596466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:09.598416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:09.688285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 984/1977] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:09 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:09.692790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:09.703738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadeaf3-2dac-4306-9331-14d47a9b222c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:09.771858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:09.772540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.772817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.773090 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.773717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:09.775692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:09.926757 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 994/1978] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:09 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:09.931160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:09.956077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256f397-c1c5-4365-a58a-e1391f51812a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:09.963098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:09.964435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:09.964435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:09.964601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:09.965221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:09.973968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:09.999945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 985/1979] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:10.003713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:10.016008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b5dc-5113-4dcb-b05d-e85eba51a845 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:10.021274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c45e0-5bce-450c-8f8d-c01c0853b0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:10.021901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c45e0-5bce-450c-8f8d-c01c0853b0ab admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:10.022184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c45e0-5bce-450c-8f8d-c01c0853b0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:10.022490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c45e0-5bce-450c-8f8d-c01c0853b0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:10.023144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c45e0-5bce-450c-8f8d-c01c0853b0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:10.024923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c45e0-5bce-450c-8f8d-c01c0853b0ab admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:10.038968 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 995/1980] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:10.043300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:10.055541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded9c2-8bbc-4cd6-8960-30022972173a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:10.062179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:10.062851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:10.063249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:10.063446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] PATH_INFO: `/v3/projects/c36e0d86e93942189945656ccb70c059/users/d625c099d9134f78b3e8940879c19106/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:10.086121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36e0d86e93942189945656ccb70c059, user_id=d625c099d9134f78b3e8940879c19106, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:10.089266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:10.153737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 986/1981] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:10 2022] PUT /identity/v3/projects/c36e0d86e93942189945656ccb70c059/users/d625c099d9134f78b3e8940879c19106/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:10.158988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:10.159275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:10.159553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:10.178659 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:10.179886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:10.198535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:10.245160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 996/1982] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:10.255658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:10.257276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:10.283137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:10.318887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bab818-9b6a-4180-a1b8-869cac455d86 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:10.322648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:10.323256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:10.323554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:10.323853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:10.333257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:10.334668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:10.344682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 987/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:12.071645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:12.071645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:12.071645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cb354-fb9a-4564-9227-fee7f520f9c0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:12.073339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:12.073339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:12.073339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:12.073339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:12.074161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:12.140111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:12.141635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9d03b-5118-4e54-88bb-53032382decf service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:12.153284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 997/1984] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:16.589002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:16.590872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:16.610194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0757a9-970f-41cc-bc61-ad11d1c0167a service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:16.616234 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:16.617302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:16.618051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:16.618543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] PATH_INFO: `/v3/users/cf3e419c53e94c05a98dcc874179526b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:16.621217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3e419c53e94c05a98dcc874179526b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:16.623822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:16.861768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3e419c53e94c05a98dcc874179526b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:16.965192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:16.965669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:16.965765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:16.995030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3e419c53e94c05a98dcc874179526b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:16.995592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3e419c53e94c05a98dcc874179526b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:16.996566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3e419c53e94c05a98dcc874179526b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:17.014159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] MFA Rules not processed for user `d625c099d9134f78b3e8940879c19106`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:17.015398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:17.021789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-d0eb51fe-06e3-418c-a537-12420cc96ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3e419c53e94c05a98dcc874179526b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:17.029098 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 988/1985] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:16 2022] DELETE /identity/v3/users/cf3e419c53e94c05a98dcc874179526b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:17.033528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.033685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.033955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.065537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:17.067612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:17.093450 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 998/1986] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:17.105736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.107696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:17.128601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213523a6-3d8e-48b6-9f7e-ba8402088ffe None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.136482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:17.207141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 989/1987] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:17.221767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.223390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:17.240243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.241169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.241455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.241732 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.242715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.244748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.279563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:17.298581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e8fff-2984-460d-88dd-acc7ea81fa9f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.302674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.303325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.303748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.304052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.315588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.317055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.332145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 990/1988] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:10:17.367351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 999/1989] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:17 2022] POST /identity/v3/projects => generated 377 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:17.368479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.379398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b322781-e018-4b0d-a194-822f14d7d34d service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.454044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.454678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.454964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.455415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.456401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.459455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.555876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 991/1990] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:17 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:17.559705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.570661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058320-ff7a-4b45-aaaf-ba5d99bb2e16 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.575398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.576112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.576395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.576670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.577370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.578954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.590516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1000/1991] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:17.595208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.605821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2d68a-13ce-4b2d-a576-1e29b64e8147 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.612664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.613349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.613629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.613899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] PATH_INFO: `/v3/projects/769cd8462dd4456fa527c611463e4677/users/74d1f362098343c7b99e2784e7944ed4/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.648915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=769cd8462dd4456fa527c611463e4677, user_id=74d1f362098343c7b99e2784e7944ed4, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.651544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.727918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 992/1992] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:17 2022] PUT /identity/v3/projects/769cd8462dd4456fa527c611463e4677/users/74d1f362098343c7b99e2784e7944ed4/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:17.732236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.744445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d731f-03aa-4846-9955-f81406da822b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.816928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.817880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.818223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.818570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.819796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.820974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.835750 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1001/1993] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:17.836526 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.848721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c6ce-505a-4592-b87f-81eb31f72f4a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.853612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:17.856383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:17.866078 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.866870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.866870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.867097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] PATH_INFO: `/v3/projects/769cd8462dd4456fa527c611463e4677/users/74d1f362098343c7b99e2784e7944ed4/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.870863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769cd8462dd4456fa527c611463e4677, user_id=74d1f362098343c7b99e2784e7944ed4, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.873926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:17.876342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75bf4d-f8a8-4d99-8f83-9ea0c664e5d2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:17.882886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:17.883541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:17.887353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:17.887353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] PATH_INFO: `/v3/projects/fc092aa40cc74450b5e652b85b616424` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:17.887353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc092aa40cc74450b5e652b85b616424)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:17.887603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 993/1994] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:17 2022] PUT /identity/v3/projects/769cd8462dd4456fa527c611463e4677/users/74d1f362098343c7b99e2784e7944ed4/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:17.887708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:18.057021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc092aa40cc74450b5e652b85b616424', 'request_id': 'req-c062aaea-3aed-438f-a964-4612251c17f7'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.163634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc092aa40cc74450b5e652b85b616424', 'request_id': 'req-c062aaea-3aed-438f-a964-4612251c17f7'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.173105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc092aa40cc74450b5e652b85b616424', 'request_id': 'req-c062aaea-3aed-438f-a964-4612251c17f7'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.197453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] The token cache is being invalidate because project fc092aa40cc74450b5e652b85b616424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:18.202153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.202153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.202153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1002/1995] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:17 2022] DELETE /identity/v3/projects/fc092aa40cc74450b5e652b85b616424 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:18.204797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:18.210640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:18.307760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b152e7-a989-40ec-9c1d-1e3d4504f063 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:18.346940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:18.375871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:18.376789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:18.377244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:18.377667 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] PATH_INFO: `/v3/users/a7ef031e0ad04e278090bb2b5e4868da` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:18.383854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ef031e0ad04e278090bb2b5e4868da)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:18.386173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:18.592253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ef031e0ad04e278090bb2b5e4868da'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.702457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ef031e0ad04e278090bb2b5e4868da'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.703140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ef031e0ad04e278090bb2b5e4868da'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.703796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ef031e0ad04e278090bb2b5e4868da'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.729053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ef031e0ad04e278090bb2b5e4868da'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:18.756656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 994/1996] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:18 2022] DELETE /identity/v3/users/a7ef031e0ad04e278090bb2b5e4868da => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:19.387139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:19.393411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:19.402789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062aaea-3aed-438f-a964-4612251c17f7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:19.536295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:19.537843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:19.538135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:19.538406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] PATH_INFO: `/v3/projects/1044a24fb8f4407895c6fa6d4d1f5aab` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:19.541323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1044a24fb8f4407895c6fa6d4d1f5aab)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:19.545922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:19.681837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1044a24fb8f4407895c6fa6d4d1f5aab', 'request_id': 'req-da992401-13c8-45e3-809f-b66a9ed72629'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:19.754585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1044a24fb8f4407895c6fa6d4d1f5aab', 'request_id': 'req-da992401-13c8-45e3-809f-b66a9ed72629'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:19.768597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1044a24fb8f4407895c6fa6d4d1f5aab', 'request_id': 'req-da992401-13c8-45e3-809f-b66a9ed72629'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:19.782929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] The token cache is being invalidate because project 1044a24fb8f4407895c6fa6d4d1f5aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:19.783483 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:19.784762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:19.798176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1003/1997] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:19 2022] DELETE /identity/v3/projects/1044a24fb8f4407895c6fa6d4d1f5aab => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:19.800116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:19.801714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:19.825063 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc97c-efb8-4b6e-8f3f-a19e3508e7b8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:19.995669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:19.996799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:19.997190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:19.997572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] PATH_INFO: `/v3/users/daad6697459643da86a056a89dd124ca` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:20.000469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daad6697459643da86a056a89dd124ca)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:20.002838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:20.256710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daad6697459643da86a056a89dd124ca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:20.325372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daad6697459643da86a056a89dd124ca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:20.326061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daad6697459643da86a056a89dd124ca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:20.326703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daad6697459643da86a056a89dd124ca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:20.333199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:20.342740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ecdc0e45-e766-4937-aea7-15d23e192808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daad6697459643da86a056a89dd124ca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:20.369650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 995/1998] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:19 2022] DELETE /identity/v3/users/daad6697459643da86a056a89dd124ca => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:21.144758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:21.146392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:21.175940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da992401-13c8-45e3-809f-b66a9ed72629 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:21.281091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:21.282959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:21.283257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:21.283708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] PATH_INFO: `/v3/projects/b375260800574b6fbe19b3409f92a5bb` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:21.285200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b375260800574b6fbe19b3409f92a5bb)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:21.287150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:21.456735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:21.471485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b375260800574b6fbe19b3409f92a5bb', 'request_id': 'req-052d8237-0828-4ebd-a404-00fa840fdf08'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:21.525309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b375260800574b6fbe19b3409f92a5bb', 'request_id': 'req-052d8237-0828-4ebd-a404-00fa840fdf08'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:21.532206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b375260800574b6fbe19b3409f92a5bb', 'request_id': 'req-052d8237-0828-4ebd-a404-00fa840fdf08'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:21.558670 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] The token cache is being invalidate because project b375260800574b6fbe19b3409f92a5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:21.558974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:21.559655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-052d8237-0828-4ebd-a404-00fa840fdf08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:21.565369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1004/1999] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:21 2022] DELETE /identity/v3/projects/b375260800574b6fbe19b3409f92a5bb => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:21.570475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:21.571020 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:21.571114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:21.632838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:21.632838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:21.846538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 996/2000] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:21.855322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539db50e-6612-410b-907e-5ac325d5c9f9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:21.855607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539db50e-6612-410b-907e-5ac325d5c9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:21.855932 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539db50e-6612-410b-907e-5ac325d5c9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:21.900490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-539db50e-6612-410b-907e-5ac325d5c9f9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:21.904885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-539db50e-6612-410b-907e-5ac325d5c9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:21.931278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1005/2001] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:21.935892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:21.957427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e2c4-6b31-4916-abfa-642de5c70dec None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:21.965246 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:21.967149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:21.967687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:21.968225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:21.969122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:21.969698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:21.971798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:21.978947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:21.979268 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:21.979506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.004608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 997/2002] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:10:22.010611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:22.025903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e765591-ef8a-482c-82a6-511e8d5cc254 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:22.046999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:22.048199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.048502 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.050101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.050101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:22.051763 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:22.054051 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:22.063046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:22.174489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1006/2003] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:10:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:22.175195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 998/2004] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:22 2022] POST /identity/v3/projects => generated 399 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:10:22.179218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:22.255422 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:22.257446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:22.342991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b95c3-14e9-4658-a0db-b105f079fcca admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:22.353805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:22.354545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.354903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.355346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.356407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:22.358660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:22.407273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12de3d-365c-4df3-aa58-eb2b2cee405c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:22.467957 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 999/2005] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:22 2022] POST /identity/v3/users => generated 300 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:22.471028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:22.495524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9308039-1b61-472f-95ca-f31cdad48881 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:22.507400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:22.508745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.509039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.509317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.510321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:22.512832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:22.533558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1000/2006] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:10:22.534930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:22.548457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7714f6-cd33-4196-a662-046cc4d5f25e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:22.549706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7714f6-cd33-4196-a662-046cc4d5f25e service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.552792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7714f6-cd33-4196-a662-046cc4d5f25e service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.553275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7714f6-cd33-4196-a662-046cc4d5f25e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.557505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b40ee3-ddc4-4f7f-8dcd-a70bb83578f5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:22.561431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:22.566821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:22.576517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.576517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.576517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.576517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:22.576517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:22.584335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1001/2007] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:22.589028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:22.603560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb076ab-1635-4d75-a483-0950c3a88012 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:22.609749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:22.610560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.611102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.612172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] PATH_INFO: `/v3/projects/e1f83285b6544f46862252510329219c/users/2a1a9ce76a6c4f90a723f86b268394fd/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.617442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:22.643189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f83285b6544f46862252510329219c, user_id=2a1a9ce76a6c4f90a723f86b268394fd, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:22.646369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:22.678542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7714f6-cd33-4196-a662-046cc4d5f25e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:22.680151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7714f6-cd33-4196-a662-046cc4d5f25e service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:22.702163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1007/2008] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:10:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 17:10:22.748443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1002/2009] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:22 2022] PUT /identity/v3/projects/e1f83285b6544f46862252510329219c/users/2a1a9ce76a6c4f90a723f86b268394fd/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:22.749261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:22.749549 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:22.749823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:22.805099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] MFA Rules not processed for user `2a1a9ce76a6c4f90a723f86b268394fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:22.806501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:23.027358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1008/2010] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:23.029677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:23.242291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea565-fcbc-4465-8884-d5f0b98a837f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:23.251309 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:23.252834 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:23.253334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:23.254058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:23.255495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:23.258274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:23.390934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:23.401072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1003/2011] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:23 2022] POST /identity/v3/projects => generated 401 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:23.411654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf49fe-24dc-4ef1-abf8-866b82063483 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:23.566208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:23.566752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:23.567026 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:23.567295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:23.567935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:23.569930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:23.697197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1009/2012] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:23 2022] POST /identity/v3/users => generated 307 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:23.701282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:23.719215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014cfc-8f43-4372-a77a-56c8f8462245 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:23.727671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:23.728669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:23.729194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:23.729672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:23.732904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:23.735312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:23.756705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1004/2013] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:23.761178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:23.781593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da1e30-2136-4fa7-b7f2-37f726100344 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:23.809213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:23.810113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:23.810418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:23.810856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] PATH_INFO: `/v3/projects/7fc66e9618224466baa4e0ee08a478a8/users/f2cb85fb56e8462f8225f1643f42cbae/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:23.844538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc66e9618224466baa4e0ee08a478a8, user_id=f2cb85fb56e8462f8225f1643f42cbae, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:23.848319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:23.927663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:23.936721 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1010/2014] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:23 2022] PUT /identity/v3/projects/7fc66e9618224466baa4e0ee08a478a8/users/f2cb85fb56e8462f8225f1643f42cbae/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:23.941490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:23.957223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b04d0a-70aa-4378-94ea-e297c9572d3b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:24.080840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd03d54-22c6-47ba-aea4-1621c18fc37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:24.081589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd03d54-22c6-47ba-aea4-1621c18fc37e admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:24.083956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd03d54-22c6-47ba-aea4-1621c18fc37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:24.084341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd03d54-22c6-47ba-aea4-1621c18fc37e admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:24.085069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd03d54-22c6-47ba-aea4-1621c18fc37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:24.096483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd03d54-22c6-47ba-aea4-1621c18fc37e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:24.103749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:24.104804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1005/2015] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:24.117922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6b05-6c2d-4317-b3bf-4f5faf1e07a2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:24.137943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:24.139366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:24.139818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:24.140338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] PATH_INFO: `/v3/projects/7fc66e9618224466baa4e0ee08a478a8/users/f2cb85fb56e8462f8225f1643f42cbae/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:24.144907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc66e9618224466baa4e0ee08a478a8, user_id=f2cb85fb56e8462f8225f1643f42cbae, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:24.155209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:24.247016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1011/2016] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:24 2022] PUT /identity/v3/projects/7fc66e9618224466baa4e0ee08a478a8/users/f2cb85fb56e8462f8225f1643f42cbae/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:24.247434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:24.247434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:24.247434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:24.335163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] MFA Rules not processed for user `f2cb85fb56e8462f8225f1643f42cbae`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:24.338306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:24.468718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:24.508446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1006/2017] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:24.518482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:24.520786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:24.645482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacb6ac-dd54-4bf4-b390-2de04f535300 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:24.651161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03462d7-232a-4422-9226-d2b248533264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:24.651161 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03462d7-232a-4422-9226-d2b248533264 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:24.651665 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03462d7-232a-4422-9226-d2b248533264 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:24.652185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03462d7-232a-4422-9226-d2b248533264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:24.664834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03462d7-232a-4422-9226-d2b248533264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:24.666501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03462d7-232a-4422-9226-d2b248533264 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:24.681319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1012/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:24 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:25.032378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:25.046872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd2840-9ed5-45f3-a43d-9aa9661a1344 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:25.067818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:25.068499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:25.068778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:25.069058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:25.157675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:25.159686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:25.171244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1007/2019] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:26.486511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:26.493297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:26.493297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:26.520421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] MFA Rules not processed for user `74d1f362098343c7b99e2784e7944ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:26.521797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:26.674078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1013/2020] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:26.683307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:26.685680 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:26.823564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70802dd8-b970-4a7d-9d3d-3ddc8c08e36c service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:26.850566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:26.851529 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:26.851975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:26.856877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:26.856877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:26.974085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:26.976737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246117f-19c7-4a03-98ec-4cb85d01fad9 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:26.992173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1008/2021] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:31.357399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:31.547880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9d03-c10b-4316-8b2c-0158550f574c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:31.557352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:31.557988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:31.558269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:31.558540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] PATH_INFO: `/v3/users/2a1a9ce76a6c4f90a723f86b268394fd` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:31.560343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1a9ce76a6c4f90a723f86b268394fd)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:31.565146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:31.585120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:31.774289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1a9ce76a6c4f90a723f86b268394fd'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:31.775394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1a9ce76a6c4f90a723f86b268394fd'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:31.896551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1a9ce76a6c4f90a723f86b268394fd'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:31.913273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1a9ce76a6c4f90a723f86b268394fd'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:31.914102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1a9ce76a6c4f90a723f86b268394fd'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:31.946875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1014/2022] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:31 2022] DELETE /identity/v3/users/2a1a9ce76a6c4f90a723f86b268394fd => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:31.951840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:31.952654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:31.952654 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:31.984148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:31.984148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:31.999183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:32.103641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1009/2023] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:32.119445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:32.121184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:32.346906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba888849-4114-49e9-bec4-1b548ed797ee admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:32.353779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:32.355010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:32.355356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:32.355918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:32.377962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:32.379591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:32.398814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1015/2024] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:10:33.162213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:33.163748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:33.209927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fc443-b551-4fb0-8def-6ff7d4534275 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:33.224191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:33.224804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:33.225088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:33.225354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] PATH_INFO: `/v3/projects/e1f83285b6544f46862252510329219c` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:33.226500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f83285b6544f46862252510329219c)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:33.228515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:33.356638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f83285b6544f46862252510329219c', 'request_id': 'req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:33.363165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f83285b6544f46862252510329219c', 'request_id': 'req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:33.370424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f83285b6544f46862252510329219c', 'request_id': 'req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:33.457899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] The token cache is being invalidate because project e1f83285b6544f46862252510329219c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:33.458581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:33.460065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:33.462432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1010/2025] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:33 2022] DELETE /identity/v3/projects/e1f83285b6544f46862252510329219c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:33.466675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:33.480244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20e983-1278-4105-96f5-62f21d6607fb service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:33.570725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:33.572671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:33.573089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:33.573468 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] PATH_INFO: `/v3/users/f2cb85fb56e8462f8225f1643f42cbae` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:33.574942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2cb85fb56e8462f8225f1643f42cbae)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:33.577253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:34.283977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:34.290421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2cb85fb56e8462f8225f1643f42cbae'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:34.291128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2cb85fb56e8462f8225f1643f42cbae'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:34.417127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2cb85fb56e8462f8225f1643f42cbae'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:34.431747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2cb85fb56e8462f8225f1643f42cbae'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:34.434219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9b2b86e8-271d-45eb-b6d2-d3e75274850a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2cb85fb56e8462f8225f1643f42cbae'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:34.457762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1016/2026] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:33 2022] DELETE /identity/v3/users/f2cb85fb56e8462f8225f1643f42cbae => generated 0 bytes in 991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:35.059242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:35.061994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:35.083148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9c60-584c-4d05-8ff3-710d61ac6a2b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:35.176472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:35.177229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:35.177513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:35.177789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] PATH_INFO: `/v3/projects/7fc66e9618224466baa4e0ee08a478a8` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:35.178936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc66e9618224466baa4e0ee08a478a8)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:35.181126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:35.354501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:35.363824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc66e9618224466baa4e0ee08a478a8', 'request_id': 'req-ba818304-f76f-4e30-8513-98fe53cab305'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:35.370545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc66e9618224466baa4e0ee08a478a8', 'request_id': 'req-ba818304-f76f-4e30-8513-98fe53cab305'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:35.378479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc66e9618224466baa4e0ee08a478a8', 'request_id': 'req-ba818304-f76f-4e30-8513-98fe53cab305'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:35.458141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] The token cache is being invalidate because project 7fc66e9618224466baa4e0ee08a478a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:35.458517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:35.459177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ba818304-f76f-4e30-8513-98fe53cab305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:35.461126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1011/2027] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:35 2022] DELETE /identity/v3/projects/7fc66e9618224466baa4e0ee08a478a8 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:35.466804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:35.468177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:35.468636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:35.513712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:35.513712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:35.600218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1017/2028] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:35.602130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:35.602418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:35.602696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:35.626639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:35.628392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:35.644667 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1012/2029] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:35.645103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:35.657410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ce7ed-4cdd-46a3-bb9b-cfa009aac3e3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:35.667237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:35.668007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:35.668318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:35.668591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:35.669321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:10:35.669632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:35.673230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:35.712078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1018/2030] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:10:35.715874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:35.727745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3ec55-38e4-4892-ba70-31ff83c24b39 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:35.733620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:35.736838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:35.736838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:35.736838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:35.736838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:35.737472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:35.855483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1013/2031] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:35 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:35.865114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:35.895617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e0e92-e622-4e4d-ac0f-8f82a351fcd1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:35.972783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:35.974257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:35.974257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:35.974717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:35.975508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:35.981234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:36.074418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1019/2032] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:35 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:36.076935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:36.094138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1b548-f114-4b15-b5cc-cae395652d9d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:36.099503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:36.100408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.100528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.100773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.101825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:36.105990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:36.121482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1014/2033] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:36.123576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:36.136063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3add2b-4f32-4ffd-bbd3-082f45f66366 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:36.141625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc7d35-ce1d-4752-a56b-393ba7b041d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:36.142257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc7d35-ce1d-4752-a56b-393ba7b041d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.142527 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc7d35-ce1d-4752-a56b-393ba7b041d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.142797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc7d35-ce1d-4752-a56b-393ba7b041d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.143443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc7d35-ce1d-4752-a56b-393ba7b041d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:36.145169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc7d35-ce1d-4752-a56b-393ba7b041d3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:36.164447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1020/2034] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:36.165599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:36.177960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc663dbe-534f-4edb-a189-d40cdf9a574c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:36.183811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:36.183811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.184150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.184555 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] PATH_INFO: `/v3/projects/3bd55cb03c474d69b82f34b7d1cef6c2/users/291d0ec7dc544ba8aa71369df23d59e4/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.211385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd55cb03c474d69b82f34b7d1cef6c2, user_id=291d0ec7dc544ba8aa71369df23d59e4, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:36.214169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:36.304467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1015/2035] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:36 2022] PUT /identity/v3/projects/3bd55cb03c474d69b82f34b7d1cef6c2/users/291d0ec7dc544ba8aa71369df23d59e4/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:36.316669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.316994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.317219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.349617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] MFA Rules not processed for user `291d0ec7dc544ba8aa71369df23d59e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:36.351174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:36.363930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:36.438591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1021/2036] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:36.439940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:36.559989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321172b6-c95a-4f0b-a0ff-100a2e1e22ce admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:36.571115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:36.571760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.572067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.572332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.573307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:36.577589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:36.693074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1016/2037] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:10:36 2022] POST /identity/v3/projects => generated 385 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:36.696937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:36.717835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69fb84-3423-4dd4-b8cd-13ecd50d091f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:36.823605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:36.824401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.824690 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.824968 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.826265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:36.828355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:36.969592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1022/2038] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:10:36 2022] POST /identity/v3/users => generated 299 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:10:36.970801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:36.982256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22124680-f7ef-4d8e-85ae-8f6232ca63a4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:36.986980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:36.987587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:36.987863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:36.988178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:36.988819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:36.990553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:37.002387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1017/2039] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:37.007023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:37.018362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221eac1-1198-43a4-87e5-c6386cb9c9c6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:37.027343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:37.028213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:37.028686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:37.029040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] PATH_INFO: `/v3/projects/0a0783cbbf134074a5b2f883bcfee4fc/users/b31968c853c04e15bea7c105dce2a971/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:37.059393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0783cbbf134074a5b2f883bcfee4fc, user_id=b31968c853c04e15bea7c105dce2a971, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:37.063042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:37.142356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:37.143287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1023/2040] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:37 2022] PUT /identity/v3/projects/0a0783cbbf134074a5b2f883bcfee4fc/users/b31968c853c04e15bea7c105dce2a971/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:10:37.154213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6a318-90e8-4fe6-9670-6fe6e9a383b2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:37.199463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:37.221024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678d515-19c0-4219-b288-c38bbae18d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:37.221645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678d515-19c0-4219-b288-c38bbae18d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:37.221989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678d515-19c0-4219-b288-c38bbae18d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:37.222325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678d515-19c0-4219-b288-c38bbae18d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:37.223036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678d515-19c0-4219-b288-c38bbae18d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:37.226205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678d515-19c0-4219-b288-c38bbae18d34 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:37.240491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1018/2041] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:10:37.241904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:37.255128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c319375-8387-4d15-a973-799bbf56d463 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:37.266423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:37.267126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:37.267409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:37.267728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] PATH_INFO: `/v3/projects/0a0783cbbf134074a5b2f883bcfee4fc/users/b31968c853c04e15bea7c105dce2a971/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:37.269837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0783cbbf134074a5b2f883bcfee4fc, user_id=b31968c853c04e15bea7c105dce2a971, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:37.272703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:37.293331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1024/2042] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:10:37 2022] PUT /identity/v3/projects/0a0783cbbf134074a5b2f883bcfee4fc/users/b31968c853c04e15bea7c105dce2a971/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:37.299742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:37.300210 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:37.300703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:37.338191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] MFA Rules not processed for user `b31968c853c04e15bea7c105dce2a971`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:37.341576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:37.423536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1019/2043] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:37.436983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:37.438679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:37.554429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f83ed-062a-4cc7-ac48-e9bbeb556310 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:37.558210 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6127b-f4ad-4aec-9799-8d327cfc115f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:37.558824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6127b-f4ad-4aec-9799-8d327cfc115f service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:37.559098 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6127b-f4ad-4aec-9799-8d327cfc115f service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:37.559363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6127b-f4ad-4aec-9799-8d327cfc115f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:37.605037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:37.647375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6127b-f4ad-4aec-9799-8d327cfc115f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:37.649088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6127b-f4ad-4aec-9799-8d327cfc115f service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:37.658552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1025/2044] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:47.431969 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=45 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:47.845008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=19 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:48.572325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:48.582953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e2e5-c879-442c-a867-87cce265d6a3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:48.586922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:48.587727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:48.589474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ad4c4-e4ea-4fd0-aa14-b71ebb85ed32 None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:48.589628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ad4c4-e4ea-4fd0-aa14-b71ebb85ed32 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:48.589881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ad4c4-e4ea-4fd0-aa14-b71ebb85ed32 None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:48.591255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1026/2045] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 17:10:48.591607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:48.592106 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:48.597399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f4a41-fa43-4b75-ad50-817a0df11662 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:48.597956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f4a41-fa43-4b75-ad50-817a0df11662 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:48.598649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f4a41-fa43-4b75-ad50-817a0df11662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:48.602821 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:48.604264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:48.614666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1020/2046] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:48.621503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6e5f4a41-fa43-4b75-ad50-817a0df11662 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:48.622926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6e5f4a41-fa43-4b75-ad50-817a0df11662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:48.734548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1027/2047] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:10:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:48.746596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:48.748190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:48.807427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e783fc-29e1-4187-bec5-19c8dd7b79c0 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:48.811067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b6ecb-48ae-4e49-8fdd-6d7d1c1bf17a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:48.811699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b6ecb-48ae-4e49-8fdd-6d7d1c1bf17a service glance] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:48.811979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b6ecb-48ae-4e49-8fdd-6d7d1c1bf17a service glance] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:48.812270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b6ecb-48ae-4e49-8fdd-6d7d1c1bf17a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:48.823351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b6ecb-48ae-4e49-8fdd-6d7d1c1bf17a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:48.825043 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b6ecb-48ae-4e49-8fdd-6d7d1c1bf17a service glance] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:48.837500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1021/2048] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:49.084577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33bc8-b914-48ca-b804-b12e779a42ee None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:49.085108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33bc8-b914-48ca-b804-b12e779a42ee None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:49.085108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33bc8-b914-48ca-b804-b12e779a42ee None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:49.086476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1028/2049] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:10:49.091299 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aad416-5b1b-4472-a321-96392f21586c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:49.091592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aad416-5b1b-4472-a321-96392f21586c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:49.091875 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aad416-5b1b-4472-a321-96392f21586c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:49.114012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-73aad416-5b1b-4472-a321-96392f21586c None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:49.115363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-73aad416-5b1b-4472-a321-96392f21586c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:49.184156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1022/2050] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:10:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:49.198118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33bc8-b914-48ca-b804-b12e779a42ee None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:49.200625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c2f33bc8-b914-48ca-b804-b12e779a42ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:49.219988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33bc8-b914-48ca-b804-b12e779a42ee None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:49.224895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2221-ebfb-4a0c-9d70-0bf1f2c3c566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:49.225768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2221-ebfb-4a0c-9d70-0bf1f2c3c566 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:49.226115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2221-ebfb-4a0c-9d70-0bf1f2c3c566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:49.226480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2221-ebfb-4a0c-9d70-0bf1f2c3c566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:49.235594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2221-ebfb-4a0c-9d70-0bf1f2c3c566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:49.237052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2221-ebfb-4a0c-9d70-0bf1f2c3c566 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:49.245661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1029/2051] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:49.330385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c0dbc-1aac-449e-ac9c-510e265e8f69 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:49.330790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c0dbc-1aac-449e-ac9c-510e265e8f69 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:49.331255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c0dbc-1aac-449e-ac9c-510e265e8f69 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:49.332744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1023/2052] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:10:49.338575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:49.338864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:49.339144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:49.366712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:49.367928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:49.378238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1030/2053] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:10:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:49.392858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c0dbc-1aac-449e-ac9c-510e265e8f69 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:49.404412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c0dbc-1aac-449e-ac9c-510e265e8f69 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:49.408698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7c63-266e-45ea-ba7d-8a5ba096ab13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:49.409361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7c63-266e-45ea-ba7d-8a5ba096ab13 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:49.409633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7c63-266e-45ea-ba7d-8a5ba096ab13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:49.409925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7c63-266e-45ea-ba7d-8a5ba096ab13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:49.422469 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b7c63-266e-45ea-ba7d-8a5ba096ab13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:49.424003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b7c63-266e-45ea-ba7d-8a5ba096ab13 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:49.436531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1024/2054] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:55.143158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:55.145120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:55.275679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f374b96-5e92-4d69-a003-e6ebf64a5cd9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:55.275679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:55.275679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:55.275679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:55.276054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] PATH_INFO: `/v3/users/211b7023c0c642f6b85b915658838a3a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:55.276054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=211b7023c0c642f6b85b915658838a3a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:55.276054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:55.742171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211b7023c0c642f6b85b915658838a3a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:55.743087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211b7023c0c642f6b85b915658838a3a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:55.871381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211b7023c0c642f6b85b915658838a3a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:55.891284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211b7023c0c642f6b85b915658838a3a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:55.892051 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211b7023c0c642f6b85b915658838a3a'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:55.894470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:55.906548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1031/2055] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:55 2022] DELETE /identity/v3/users/211b7023c0c642f6b85b915658838a3a => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:55.911026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:55.911683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:55.911683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:55.933189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:55.934443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:55.998223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1025/2056] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:10:56.008767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:56.010151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:56.069531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cf89-9e7b-43a0-8b4b-2f27dc5ba34b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:56.073331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663ae31-7731-4b6c-bb55-551059b98dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:56.073989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ae31-7731-4b6c-bb55-551059b98dcb service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:56.074271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ae31-7731-4b6c-bb55-551059b98dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:56.074541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ae31-7731-4b6c-bb55-551059b98dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:56.084076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663ae31-7731-4b6c-bb55-551059b98dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:56.085501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663ae31-7731-4b6c-bb55-551059b98dcb service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:56.095424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1032/2057] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:56.600755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:56.606986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:56.614441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30fade-5028-4bce-a1c1-6317da7e180b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:56.619111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:56.619747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:56.620053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:56.620384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] PATH_INFO: `/v3/projects/9ea1d9bac2d548fea62e9d540a5fbcc0` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:56.621479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea1d9bac2d548fea62e9d540a5fbcc0)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:56.623455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:56.650003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:56.729489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea1d9bac2d548fea62e9d540a5fbcc0', 'request_id': 'req-b52d0856-75d4-4190-9212-a43f012fad81'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:56.735806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea1d9bac2d548fea62e9d540a5fbcc0', 'request_id': 'req-b52d0856-75d4-4190-9212-a43f012fad81'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:56.742437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea1d9bac2d548fea62e9d540a5fbcc0', 'request_id': 'req-b52d0856-75d4-4190-9212-a43f012fad81'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:56.824962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] The token cache is being invalidate because project 9ea1d9bac2d548fea62e9d540a5fbcc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:56.825344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:56.829014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:56.830908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1026/2058] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:56 2022] DELETE /identity/v3/projects/9ea1d9bac2d548fea62e9d540a5fbcc0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:57.657595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:57.658084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:57.658084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:57.677525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:57.678729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:57.748014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1033/2059] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:10:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:57.760642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:57.762757 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:57.836671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d0856-75d4-4190-9212-a43f012fad81 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:57.842634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89186fee-7150-4b95-b572-c456d7a40055 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:57.842634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89186fee-7150-4b95-b572-c456d7a40055 service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:57.842634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89186fee-7150-4b95-b572-c456d7a40055 service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:57.843493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89186fee-7150-4b95-b572-c456d7a40055 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:57.851886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89186fee-7150-4b95-b572-c456d7a40055 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:57.853507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89186fee-7150-4b95-b572-c456d7a40055 service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:57.863849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1027/2060] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:10:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 17:10:58.132283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:58.134045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:58.278735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd4ee-9182-4751-a73c-608f3096f258 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:58.289046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:58.294419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:58.294419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:58.294419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] PATH_INFO: `/v3/users/291d0ec7dc544ba8aa71369df23d59e4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:58.294419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291d0ec7dc544ba8aa71369df23d59e4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:58.297577 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:58.506309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:58.512605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291d0ec7dc544ba8aa71369df23d59e4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:58.513440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291d0ec7dc544ba8aa71369df23d59e4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:58.579672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291d0ec7dc544ba8aa71369df23d59e4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:58.595461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291d0ec7dc544ba8aa71369df23d59e4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:58.597765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291d0ec7dc544ba8aa71369df23d59e4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:58.616888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1034/2061] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:58 2022] DELETE /identity/v3/users/291d0ec7dc544ba8aa71369df23d59e4 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:58.623229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:58.623229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:58.623229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:58.666600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:10:58.671404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:58.755308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1028/2062] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:10:58.765459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:58.767072 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:58.829481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8b28-b8ab-4e43-87cf-2e23cb927e71 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:58.833470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:58.834198 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:58.834545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:58.834905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:58.845508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:58.849142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:58.876769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1035/2063] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:10:59.407833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:59.409418 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:59.426202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d977ced-42b7-477e-aa6e-17fac1a28395 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:59.431165 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:59.431811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:59.432122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:59.432402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] PATH_INFO: `/v3/projects/3bd55cb03c474d69b82f34b7d1cef6c2` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:59.433682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd55cb03c474d69b82f34b7d1cef6c2)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:59.435604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:59.596710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:10:59.607870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd55cb03c474d69b82f34b7d1cef6c2', 'request_id': 'req-8569a1a2-693f-483f-9771-73def7f825ba'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:59.614282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd55cb03c474d69b82f34b7d1cef6c2', 'request_id': 'req-8569a1a2-693f-483f-9771-73def7f825ba'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:59.620948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd55cb03c474d69b82f34b7d1cef6c2', 'request_id': 'req-8569a1a2-693f-483f-9771-73def7f825ba'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:59.722371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] The token cache is being invalidate because project 3bd55cb03c474d69b82f34b7d1cef6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:10:59.722662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:59.723308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:10:59.724958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1029/2064] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:10:59 2022] DELETE /identity/v3/projects/3bd55cb03c474d69b82f34b7d1cef6c2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:10:59.729118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:59.739648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c255dd-6298-4732-9241-0d8c8f9b13a0 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:59.800062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:59.800873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:10:59.801125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:10:59.801403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] PATH_INFO: `/v3/users/b31968c853c04e15bea7c105dce2a971` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:10:59.802533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31968c853c04e15bea7c105dce2a971)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:10:59.804602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:10:59.972215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:10:59.973660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:10:59.993168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569a1a2-693f-483f-9771-73def7f825ba admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:10:59.998883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:10:59.999705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:00.000562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:00.001267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] PATH_INFO: `/v3/users/99aaa7ac95a545178fb3ee8b7877b74f` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:00.002593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99aaa7ac95a545178fb3ee8b7877b74f)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:00.004813 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:00.036412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31968c853c04e15bea7c105dce2a971'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.037219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31968c853c04e15bea7c105dce2a971'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.134501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31968c853c04e15bea7c105dce2a971'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.143395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:00.149286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31968c853c04e15bea7c105dce2a971'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.150121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ca9db9fc-844e-41f0-a41c-c0c25de2a16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31968c853c04e15bea7c105dce2a971'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.164535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1036/2065] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:59 2022] DELETE /identity/v3/users/b31968c853c04e15bea7c105dce2a971 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:11:00.224681 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99aaa7ac95a545178fb3ee8b7877b74f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.301325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99aaa7ac95a545178fb3ee8b7877b74f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.302156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99aaa7ac95a545178fb3ee8b7877b74f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.305739 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99aaa7ac95a545178fb3ee8b7877b74f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.319714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99aaa7ac95a545178fb3ee8b7877b74f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:00.340855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1030/2066] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:10:59 2022] DELETE /identity/v3/users/99aaa7ac95a545178fb3ee8b7877b74f => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:11:00.345797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:00.346095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:00.346367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:00.383630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:00.385187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:00.517418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1037/2067] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:00.524714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:00.526908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:00.622919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304958dc-54fe-4b6e-935e-6087997a2369 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:00.627475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:00.628731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:00.628731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:00.628975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:00.652534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:00.652534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:00.689119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1031/2068] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:11:00.736307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:00.738960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:00.774473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98670f4-a4d4-4bde-aecb-788c8967467f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:00.786288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:00.788791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:00.789900 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:00.790380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] PATH_INFO: `/v3/projects/0a0783cbbf134074a5b2f883bcfee4fc` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:00.792606 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0783cbbf134074a5b2f883bcfee4fc)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:00.794916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:00.943366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0783cbbf134074a5b2f883bcfee4fc', 'request_id': 'req-bf37ec29-81a7-4913-95e4-5c413817ac9c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.050218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0783cbbf134074a5b2f883bcfee4fc', 'request_id': 'req-bf37ec29-81a7-4913-95e4-5c413817ac9c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.057233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0783cbbf134074a5b2f883bcfee4fc', 'request_id': 'req-bf37ec29-81a7-4913-95e4-5c413817ac9c'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.077254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] The token cache is being invalidate because project 0a0783cbbf134074a5b2f883bcfee4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:11:01.077550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.078213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-bf37ec29-81a7-4913-95e4-5c413817ac9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.080015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1038/2069] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:11:00 2022] DELETE /identity/v3/projects/0a0783cbbf134074a5b2f883bcfee4fc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:11:01.181981 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:01.183452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:01.221002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbca0dc-2292-4ba2-87a3-cb53143907a2 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:01.286688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:01.311670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:01.312326 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:01.312608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:01.312880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] PATH_INFO: `/v3/projects/bb22ce3005aa43b09352e92e64ae2122` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:01.313989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb22ce3005aa43b09352e92e64ae2122)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:01.316343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:01.424767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb22ce3005aa43b09352e92e64ae2122', 'request_id': 'req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.430382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb22ce3005aa43b09352e92e64ae2122', 'request_id': 'req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.437179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb22ce3005aa43b09352e92e64ae2122', 'request_id': 'req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.526270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] The token cache is being invalidate because project bb22ce3005aa43b09352e92e64ae2122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:11:01.526995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.527616 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1f244c04-c87e-44de-8d42-2a3d7ebcb2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:01.529220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1032/2070] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:11:01 2022] DELETE /identity/v3/projects/bb22ce3005aa43b09352e92e64ae2122 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:11:01.534457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:01.534746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:01.535025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:01.559032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:01.560266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:01.627693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1039/2071] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:01.637776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:01.638110 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:01.640417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:01.670693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:01.672970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:01.704272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1033/2072] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:01.705624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:01.713226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:01.717204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee16cfc-3b89-44ba-89ab-cb9d1ec4f43d None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:01.722697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:01.723526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:01.723973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:01.724451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:01.725271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:11:01.725712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:01.727444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:01.740105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1040/2073] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:11:01.744631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:01.757226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72025f20-5448-4043-a96f-573eafd4ac3d None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:01.761963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:01.762573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:01.762847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:01.763115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:01.763743 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:01.766334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:01.898481 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1034/2074] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:11:01 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:11:01.899533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:01.915151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e4f12-6d3e-451c-8133-07be29de6f7a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:01.998129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:01.999637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:01.999915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:02.000217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:02.000855 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:02.002772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:02.085237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1041/2075] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:11:01 2022] POST /identity/v3/users => generated 282 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:11:02.091520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:02.102310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd908339-2ab6-4529-9be6-2d3628c96965 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:02.106859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:02.107441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:02.107719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:02.107986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:02.108684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:02.110254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:02.120945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1035/2076] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:11:02.125250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:02.135718 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1a5ae-6beb-41b1-98f2-bbe4cfcfb9d1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:02.141342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0230ef-4530-4054-9165-ea8f3eac80ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:02.141981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0230ef-4530-4054-9165-ea8f3eac80ec admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:02.142244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0230ef-4530-4054-9165-ea8f3eac80ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:02.142534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0230ef-4530-4054-9165-ea8f3eac80ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:02.143167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0230ef-4530-4054-9165-ea8f3eac80ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:02.144789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0230ef-4530-4054-9165-ea8f3eac80ec admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:02.157151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1042/2077] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:11:02.161492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:02.171686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d3e96-d6d5-4da8-8558-811c023e83ef admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:02.176374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:02.177021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:02.177302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:02.177576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] PATH_INFO: `/v3/projects/d41c7edfc3ff4da0a63eb6d4edf548b0/users/7dbb3f0d81734bc6a20fde86a4fe8cf2/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:02.192076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41c7edfc3ff4da0a63eb6d4edf548b0, user_id=7dbb3f0d81734bc6a20fde86a4fe8cf2, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:02.194697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:02.277719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1036/2078] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:11:02 2022] PUT /identity/v3/projects/d41c7edfc3ff4da0a63eb6d4edf548b0/users/7dbb3f0d81734bc6a20fde86a4fe8cf2/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:11:02.288660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5a5d-0525-4b3d-86c2-912539fbcc51 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:02.288945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5a5d-0525-4b3d-86c2-912539fbcc51 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:02.289225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5a5d-0525-4b3d-86c2-912539fbcc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:02.322077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-143c5a5d-0525-4b3d-86c2-912539fbcc51 None None] MFA Rules not processed for user `7dbb3f0d81734bc6a20fde86a4fe8cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:02.323354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-143c5a5d-0525-4b3d-86c2-912539fbcc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:02.399206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1043/2079] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:02.407785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:02.409570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:02.471426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bfece-fbb2-49f0-ae51-2230afe0340c admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:02.479420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:02.480544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:02.480880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:02.481397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:02.492399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:02.493979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:02.505937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1037/2080] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:11:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:11:03.756303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99aa13-c1d9-441c-9853-24d7cff37711 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:03.757010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99aa13-c1d9-441c-9853-24d7cff37711 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:03.757010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99aa13-c1d9-441c-9853-24d7cff37711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:03.777155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-3c99aa13-c1d9-441c-9853-24d7cff37711 None None] MFA Rules not processed for user `d625c099d9134f78b3e8940879c19106`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:03.779056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3c99aa13-c1d9-441c-9853-24d7cff37711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:03.839200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1044/2081] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:12.499962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:13.831542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:25.356915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:25.358355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:25.418788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af24f52-b92c-44bd-a554-fd5fd645961f service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:25.422482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:25.423049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:25.423322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:25.423588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:25.425430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:25.489998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:25.491415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:25.503425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1038/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:11:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:11:26.560009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef4f22-85a7-4b2a-a3fd-64721357618e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:26.560621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef4f22-85a7-4b2a-a3fd-64721357618e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:26.560920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef4f22-85a7-4b2a-a3fd-64721357618e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:26.585773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6aef4f22-85a7-4b2a-a3fd-64721357618e None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:26.586912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6aef4f22-85a7-4b2a-a3fd-64721357618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:26.650484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1045/2083] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 10 17:11:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:26.673837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:26.675304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:26.757261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3fd3e-6684-457b-9e23-33f64449047f service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:26.762345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:26.762955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:26.763227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:26.763522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:26.772370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:26.773949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:26.782791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1039/2084] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:11:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:11:36.644188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=25 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:36.778823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=82 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:41.933534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:41.934024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:41.934120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:41.955743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] MFA Rules not processed for user `1e8d2e4a8a334eea973280b3819219e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:41.956930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:42.011727 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1046/2085] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:43.951721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:43.953178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:44.022446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58cd46-f9e0-4f88-bcee-f7da3912e1f6 service swift] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:44.027655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:44.028292 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:44.028571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:44.028841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:44.030554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:44.041887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:44.043436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775c4d2-4dd0-4613-9332-7e7b931e2380 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:44.053740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1040/2086] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:11:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:11:45.948645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:46.010365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029aff7-019f-4935-9a5e-99499c3db069 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:46.014829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:46.015533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:46.020321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:46.020321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] PATH_INFO: `/v3/users/7dbb3f0d81734bc6a20fde86a4fe8cf2` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:46.020321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dbb3f0d81734bc6a20fde86a4fe8cf2)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:46.020321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:46.212299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dbb3f0d81734bc6a20fde86a4fe8cf2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:46.212987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dbb3f0d81734bc6a20fde86a4fe8cf2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:46.272197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dbb3f0d81734bc6a20fde86a4fe8cf2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:46.286298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dbb3f0d81734bc6a20fde86a4fe8cf2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:46.287029 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dbb3f0d81734bc6a20fde86a4fe8cf2'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:46.300739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1047/2087] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:11:45 2022] DELETE /identity/v3/users/7dbb3f0d81734bc6a20fde86a4fe8cf2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:11:46.304940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:46.305382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:46.305500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:46.324842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:11:46.326044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:46.391836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1041/2088] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:11:46.405003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:46.406672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:46.439266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:46.471964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d03025-c76f-4371-b403-eaa1db017a6b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:46.476333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d7ec0-bebe-4825-b267-ed0f83d72efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:46.477007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d7ec0-bebe-4825-b267-ed0f83d72efa service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:46.477293 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d7ec0-bebe-4825-b267-ed0f83d72efa service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:46.477620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d7ec0-bebe-4825-b267-ed0f83d72efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:46.486840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d7ec0-bebe-4825-b267-ed0f83d72efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:46.488318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d7ec0-bebe-4825-b267-ed0f83d72efa service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:46.498279 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1048/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:11:46.902290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:11:46.904009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:11:46.919158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b4a1-c8ac-4588-9ccc-f3af403c38b7 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:11:46.924553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:11:46.925325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:11:46.926185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:11:46.926492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] PATH_INFO: `/v3/projects/d41c7edfc3ff4da0a63eb6d4edf548b0` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:11:46.927829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41c7edfc3ff4da0a63eb6d4edf548b0)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:11:46.929800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:11:47.052322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41c7edfc3ff4da0a63eb6d4edf548b0', 'request_id': 'req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:47.057781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41c7edfc3ff4da0a63eb6d4edf548b0', 'request_id': 'req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:47.064602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41c7edfc3ff4da0a63eb6d4edf548b0', 'request_id': 'req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:47.172476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] The token cache is being invalidate because project d41c7edfc3ff4da0a63eb6d4edf548b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:11:47.172772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:47.174452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0fb40da8-bf79-4c4d-aa5f-ecf7407ef54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:11:47.180832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1042/2090] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:11:46 2022] DELETE /identity/v3/projects/d41c7edfc3ff4da0a63eb6d4edf548b0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:11:56.600575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=15 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:11:57.169104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:12:30.252720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:12:30.253271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:12:30.253409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:12:30.273815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] MFA Rules not processed for user `b8507de448b6455fb792bda0c949421d`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:12:30.275002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:12:30.329215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1049/2091] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:12:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:12:30.984971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:12:30.985476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:12:30.985576 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:12:31.004895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] MFA Rules not processed for user `8bb27ad53d4a454eb10268ab27ad3258`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:12:31.006080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:12:31.081580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1043/2092] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:12:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:12:40.322636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=25 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:12:41.069870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=25 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:13:06.400853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:06.402253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:06.458099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42285246-e480-4fe1-9f09-7ae18305853c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:06.461707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:06.467735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:06.467735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:06.467735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:06.467735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:06.534770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:06.536207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa377b-cad2-4dc9-b8df-73e9cb6913b5 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:06.546948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1050/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:07.725180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:07.735099 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb775a-06c6-497f-a71a-45086874f70b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:07.738660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:07.739252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:07.739537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:07.739806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:07.741550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:07.754370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:07.755767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:07.766945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1044/2094] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:13.749396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:13.749949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:13.749949 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:13.769112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] MFA Rules not processed for user `1e8d2e4a8a334eea973280b3819219e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:13:13.770282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:13.822756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1051/2095] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:13:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:13:17.762328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=6 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:13:20.468697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:20.470115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:20.483635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67247371-c22d-4edf-8ce3-eab745a216b1 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:20.489173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:20.489761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:20.490040 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:20.490311 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:20.492511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:20.502355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:20.503898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415770a-d314-41c7-b9ff-c723fea12241 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:20.513317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1045/2096] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:23.131923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:23.144756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c2730-6925-4b44-b01a-d9aae879ecf8 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:23.151181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:23.151819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:23.152138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:23.152430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:23.154219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:23.167344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:23.167344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:23.177225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1052/2097] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:30.509181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=6 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:13:33.172681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=85 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:13:36.971973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767cfba-15b4-4c77-adce-09496ddb5e82 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:36.973355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767cfba-15b4-4c77-adce-09496ddb5e82 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:36.973705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767cfba-15b4-4c77-adce-09496ddb5e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:36.998553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a767cfba-15b4-4c77-adce-09496ddb5e82 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:13:36.999760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a767cfba-15b4-4c77-adce-09496ddb5e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:37.057633 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1046/2098] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:13:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:13:37.181679 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:37.183167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:37.252419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc962-89f2-4754-b283-2891cb217fe0 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:37.257424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:37.258025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:37.258289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:37.258561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:37.266538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:37.268138 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:37.276043 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1053/2099] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:13:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 17:13:42.869134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:42.869134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:42.869134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:42.902406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] MFA Rules not processed for user `d625c099d9134f78b3e8940879c19106`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:13:42.903623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:42.987553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1047/2100] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:13:43.316291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:43.325743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d5b19-8117-45d1-998b-99d37cb72fc0 service swift] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:43.329436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:43.329942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:43.330258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:43.330538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:43.332259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:43.341482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:43.342943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:43.351599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1054/2101] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:44.165464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:44.168826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:44.307188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380f73-50ca-4369-b1e9-4a98202c9a71 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:44.312948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:44.313700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:44.314079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:44.314453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:44.316750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:44.330303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:44.331947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:44.343217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1048/2102] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:49.102564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:49.106188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:49.120102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc93750-4122-4d4b-8f8a-369474561833 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:49.122793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:49.123950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:49.124563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:49.125040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:49.140517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:49.161449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:49.161449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:49.169151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1055/2103] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:49.922929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:49.928994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:49.948886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9955d-d3a4-4bf0-8b8b-0da36d9a619f service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:49.961831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:49.962465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:49.962748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:49.963057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:49.964933 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:49.979765 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:49.981609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:49.993495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1049/2104] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:52.904408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:52.918256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ebd0-0b61-4281-bd56-0d9c52273d0c service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:52.928430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:52.928430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:52.928430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:52.928430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:52.929367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:52.941133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:52.942689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:52.953743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1056/2105] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:13:56.866005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:13:56.878329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dc214-266f-4af9-ad28-d6fad9e217db service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:13:56.882538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:13:56.883193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:13:56.883474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:13:56.883749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:13:56.885806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:13:56.895490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:13:56.896940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b7b63-a055-4514-a8f2-95dd6e966439 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:13:56.905539 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1050/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:13:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:01.152662 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:01.162766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b8d89-3941-4bd9-bfde-ff29768c5d91 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:01.166677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:01.167281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:01.167623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:01.168117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:01.169954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:01.232735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:01.234194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:01.243857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1057/2107] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:04.859093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db2726-670b-434b-92b4-98da620f880a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:04.859602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db2726-670b-434b-92b4-98da620f880a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:04.859685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db2726-670b-434b-92b4-98da620f880a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:04.867992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:04.885882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a8db2726-670b-434b-92b4-98da620f880a None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:04.887357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a8db2726-670b-434b-92b4-98da620f880a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:04.909939 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1051/2108] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:14:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:04.928444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:04.935701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=81 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:04.948277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b5801-26b0-4816-a7f7-c5944af8c851 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:04.957030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e77393-d378-455a-af27-a1dde44819e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:04.957888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77393-d378-455a-af27-a1dde44819e8 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:04.958165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77393-d378-455a-af27-a1dde44819e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:04.958590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77393-d378-455a-af27-a1dde44819e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:04.969048 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e77393-d378-455a-af27-a1dde44819e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:04.970793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e77393-d378-455a-af27-a1dde44819e8 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:04.980703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1058/2109] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:14:05.356434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb99471-c129-4e92-b001-cf1a04fb63b2 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:05.356995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb99471-c129-4e92-b001-cf1a04fb63b2 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:05.357444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb99471-c129-4e92-b001-cf1a04fb63b2 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:05.359676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1052/2110] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 10 17:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:14:05.365025 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:05.366418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:05.366939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:05.395884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] MFA Rules not processed for user `d882630eb293423e84ecbd0c63fb6733`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:05.398722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:05.424104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1059/2111] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 10 17:14:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:05.424921 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb99471-c129-4e92-b001-cf1a04fb63b2 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:05.440083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb99471-c129-4e92-b001-cf1a04fb63b2 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:05.446384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:05.447883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:05.461722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:05.462006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:05.464284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:05.476639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:05.479384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:05.492334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1053/2112] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:05.523157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:05.528378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:05.544118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215e9c2-9d53-4f9b-8f94-e39c0f2f5cae None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:05.552014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:05.553305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:05.557080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:05.557080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] PATH_INFO: `/v3/users/8bb27ad53d4a454eb10268ab27ad3258` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:05.557080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb27ad53d4a454eb10268ab27ad3258)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:05.559223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:05.774057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb27ad53d4a454eb10268ab27ad3258'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:05.774916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb27ad53d4a454eb10268ab27ad3258'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:05.845106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb27ad53d4a454eb10268ab27ad3258'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:05.865465 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb27ad53d4a454eb10268ab27ad3258'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:05.866332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3450094d-9331-4336-b592-0db56ef032cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb27ad53d4a454eb10268ab27ad3258'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:05.884209 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1060/2113] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:05 2022] DELETE /identity/v3/users/8bb27ad53d4a454eb10268ab27ad3258 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:06.167861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:06.184626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e26118-d25e-4d51-a12e-2f93fb328c4b service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:06.308468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:06.309182 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:06.309544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:06.309861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] PATH_INFO: `/v3/projects/18aef1375628417b91f816bccd05d1cf` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:06.311715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=18aef1375628417b91f816bccd05d1cf)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:06.314142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:06.513867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18aef1375628417b91f816bccd05d1cf', 'request_id': 'req-66561303-66d0-455b-958e-8d23c91293ec'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:06.529911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18aef1375628417b91f816bccd05d1cf', 'request_id': 'req-66561303-66d0-455b-958e-8d23c91293ec'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:06.543714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18aef1375628417b91f816bccd05d1cf', 'request_id': 'req-66561303-66d0-455b-958e-8d23c91293ec'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:06.640890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] The token cache is being invalidate because project 18aef1375628417b91f816bccd05d1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:06.641194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:06.641971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-66561303-66d0-455b-958e-8d23c91293ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:06.643733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1054/2114] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:06 2022] DELETE /identity/v3/projects/18aef1375628417b91f816bccd05d1cf => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:06.648680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:06.649142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:06.649345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:06.671165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:06.672420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:06.737336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1061/2115] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:06.745931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:06.746306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:06.746472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:06.767224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:06.768468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:06.776532 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1055/2116] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:06.780874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:06.790785 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067b128-84e5-4836-af01-bc87ea198bd5 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:06.795653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:06.796301 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:06.796599 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:06.796843 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:06.797524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:06.797826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:06.799411 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:06.813778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1062/2117] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:14:06.817637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:06.829863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827758a-c5a0-41d4-bce5-c287f17cb535 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:06.834928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:06.835696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:06.835976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:06.836312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:06.836919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:06.838873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:06.970653 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1056/2118] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:06 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:06.975066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:06.986516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c0b61-476a-468d-8e2a-c4902ab62145 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:07.033555 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:07.050392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:07.051005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.051282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.051620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.052278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:07.054260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:07.162798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1063/2119] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:06 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:07.167364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:07.183896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2db7b-5167-480c-bcdd-631cdadc0bf9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:07.188925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:07.189618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.189971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.190314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.191066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:07.192920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:07.205090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1057/2120] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:07.208945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:07.219676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e3a6-c876-42ed-ac47-d860307f3913 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:07.227141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0ca61-1909-430b-a80a-b17d64e96a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:07.227788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0ca61-1909-430b-a80a-b17d64e96a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.231644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0ca61-1909-430b-a80a-b17d64e96a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.231963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0ca61-1909-430b-a80a-b17d64e96a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.232885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0ca61-1909-430b-a80a-b17d64e96a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:07.234801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0ca61-1909-430b-a80a-b17d64e96a0a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:07.252287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1064/2121] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:07.252795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:07.270232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c85c20-a14d-4305-9a10-42f53efe0389 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:07.279695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:07.282999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.282999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.282999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] PATH_INFO: `/v3/projects/265d5f2144c54c43af183f79aab75b79/users/06e133bcec2d466ab4a8690abbe6a3ed/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.298830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=265d5f2144c54c43af183f79aab75b79, user_id=06e133bcec2d466ab4a8690abbe6a3ed, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:07.302464 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:07.396070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1058/2122] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:07 2022] PUT /identity/v3/projects/265d5f2144c54c43af183f79aab75b79/users/06e133bcec2d466ab4a8690abbe6a3ed/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:07.408784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.409111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.409307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.447244 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] MFA Rules not processed for user `06e133bcec2d466ab4a8690abbe6a3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:07.448649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:07.565839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1065/2123] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:07.566403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:07.589730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:07.716268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c7723-ee44-4c6f-93a2-6809b727c2ab admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:07.728472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:07.729695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.729989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.730336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.730928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:07.733026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:07.885788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:07.891601 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1059/2124] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:07 2022] POST /identity/v3/projects => generated 393 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:07.897886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c195ff2-68b5-4051-a0e1-39f61507ec20 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:07.969037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:07.969684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:07.970036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:07.970379 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:07.971085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:07.973200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.057363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1066/2125] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:07 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:08.061580 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:08.073172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8a51-62c4-41a5-94b2-706b2fe53935 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:08.078019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:08.078626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.078908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.079208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.079853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:08.081466 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.094149 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1060/2126] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:08.098438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:08.108941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634864c7-012b-42c2-b0d5-60c406a9444d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:08.114553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:08.115274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.115542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.115815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] PATH_INFO: `/v3/projects/1ab4f53030974c828ecf83bf582f5eba/users/ebccb1bc0a2049a2bbf240fccef12652/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.132316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab4f53030974c828ecf83bf582f5eba, user_id=ebccb1bc0a2049a2bbf240fccef12652, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:08.136992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.254956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1067/2127] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:08 2022] PUT /identity/v3/projects/1ab4f53030974c828ecf83bf582f5eba/users/ebccb1bc0a2049a2bbf240fccef12652/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:08.260413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:08.273276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ad47a-a5b9-4a77-8666-dcb689c4bcce admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:08.335788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bb79b-dae5-4679-82e6-371090e383ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:08.339389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bb79b-dae5-4679-82e6-371090e383ba admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.339389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bb79b-dae5-4679-82e6-371090e383ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.339389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bb79b-dae5-4679-82e6-371090e383ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.339389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bb79b-dae5-4679-82e6-371090e383ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:08.339809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bb79b-dae5-4679-82e6-371090e383ba admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.352206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1061/2128] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:08.356609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:08.371176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395438e1-6e85-49dc-8ec8-90d03e831b5f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:08.376252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:08.376913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.377195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.377469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] PATH_INFO: `/v3/projects/1ab4f53030974c828ecf83bf582f5eba/users/ebccb1bc0a2049a2bbf240fccef12652/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.379674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab4f53030974c828ecf83bf582f5eba, user_id=ebccb1bc0a2049a2bbf240fccef12652, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:08.382330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.396617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1068/2129] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:08 2022] PUT /identity/v3/projects/1ab4f53030974c828ecf83bf582f5eba/users/ebccb1bc0a2049a2bbf240fccef12652/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:08.408615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.412084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.412084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.442903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] MFA Rules not processed for user `ebccb1bc0a2049a2bbf240fccef12652`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:08.444670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:08.522498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1062/2130] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:08.533608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:08.535623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:08.542160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:08.602130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f486e-5e62-4c7d-80cc-77637340bdf5 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:08.607034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:08.607778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.608113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.608388 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.618018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:08.620277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.631866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1069/2131] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:08.683889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:08.685393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:08.693545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:08.698731 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708849f1-df44-4a93-a18e-88de3617d055 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:08.705207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:08.705956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:08.706446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:08.706819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:08.800623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:08.802564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:08.826599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1063/2132] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:14.594976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:14.597260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:14.697752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70230b-5de0-4b46-852d-78fbd95a42a0 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:14.703258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:14.703876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:14.704233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:14.704486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:14.706212 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:14.719396 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:14.721030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691486f-ef42-4f4c-92dd-56ebd1936145 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:14.731924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1070/2133] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:14.815478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:14.891596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafae811-e748-40f5-96bc-70ef4cbad249 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:14.896312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:14.896954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:14.897231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:14.897504 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] PATH_INFO: `/v3/users/06e133bcec2d466ab4a8690abbe6a3ed` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:14.898781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e133bcec2d466ab4a8690abbe6a3ed)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:14.900749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:15.044251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e133bcec2d466ab4a8690abbe6a3ed'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:15.106703 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e133bcec2d466ab4a8690abbe6a3ed'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:15.107323 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e133bcec2d466ab4a8690abbe6a3ed'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:15.107934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e133bcec2d466ab4a8690abbe6a3ed'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:15.121554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e133bcec2d466ab4a8690abbe6a3ed'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:15.134902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1064/2134] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:14 2022] DELETE /identity/v3/users/06e133bcec2d466ab4a8690abbe6a3ed => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:15.139439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:15.139767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:15.140061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:15.158827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:15.160038 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:15.221879 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1071/2135] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:15.232207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:15.233608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:15.261902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:15.293347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba0206-e32c-40bf-aa7a-b65734955c97 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:15.297343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:15.297891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:15.298169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:15.298441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:15.309756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:15.311229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:15.321126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1065/2136] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:15.914993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:15.916849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:15.939467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da99ad9-08ed-4690-b057-2267050ff3d7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:15.945117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:15.945764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:15.946040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:15.946365 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] PATH_INFO: `/v3/projects/265d5f2144c54c43af183f79aab75b79` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:15.947980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265d5f2144c54c43af183f79aab75b79)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:15.949915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:16.080835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:16.097474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265d5f2144c54c43af183f79aab75b79', 'request_id': 'req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.164418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265d5f2144c54c43af183f79aab75b79', 'request_id': 'req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.170195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265d5f2144c54c43af183f79aab75b79', 'request_id': 'req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.181305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] The token cache is being invalidate because project 265d5f2144c54c43af183f79aab75b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:16.191819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.191819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.191819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1072/2137] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:15 2022] DELETE /identity/v3/projects/265d5f2144c54c43af183f79aab75b79 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:16.192176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:16.198388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967414b-f838-4856-92b4-1780ab7c2c78 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:16.259998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:16.263919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:16.263919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:16.263919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] PATH_INFO: `/v3/users/ebccb1bc0a2049a2bbf240fccef12652` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:16.263919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebccb1bc0a2049a2bbf240fccef12652)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:16.264567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:16.440628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebccb1bc0a2049a2bbf240fccef12652'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.507772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebccb1bc0a2049a2bbf240fccef12652'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.508437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebccb1bc0a2049a2bbf240fccef12652'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.509084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebccb1bc0a2049a2bbf240fccef12652'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.522221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5125386e-bb11-4fdc-8549-a65f9cd46cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebccb1bc0a2049a2bbf240fccef12652'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:16.534879 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1066/2138] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:16 2022] DELETE /identity/v3/users/ebccb1bc0a2049a2bbf240fccef12652 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:17.092652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:17.094232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:17.108764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8421dd-12ec-4ee9-a4f4-5951d5cd7302 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:17.174060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:17.175155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.175155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.175337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] PATH_INFO: `/v3/projects/1ab4f53030974c828ecf83bf582f5eba` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.176403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab4f53030974c828ecf83bf582f5eba)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:17.178303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:17.315147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab4f53030974c828ecf83bf582f5eba', 'request_id': 'req-3be90769-a099-48c3-bb7e-dea9f94d2b48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:17.381865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab4f53030974c828ecf83bf582f5eba', 'request_id': 'req-3be90769-a099-48c3-bb7e-dea9f94d2b48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:17.388343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab4f53030974c828ecf83bf582f5eba', 'request_id': 'req-3be90769-a099-48c3-bb7e-dea9f94d2b48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:17.400501 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] The token cache is being invalidate because project 1ab4f53030974c828ecf83bf582f5eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:17.400792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:17.401453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3be90769-a099-48c3-bb7e-dea9f94d2b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:17.403000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1073/2139] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:17 2022] DELETE /identity/v3/projects/1ab4f53030974c828ecf83bf582f5eba => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:17.408317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.408777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.408915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.429772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:17.430966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:17.479621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:17.505017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1067/2140] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:17.513515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.513515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.513787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.535298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:17.541295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:17.545384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1074/2141] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:17.549641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:17.563194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57430173-4afd-40b4-811a-1bc3c037dd62 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:17.569727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:17.570156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.570436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.570887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.572552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:17.572867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:17.574511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:17.587225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1068/2142] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:14:17.592561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:17.605003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ade6-35e3-4c2e-9307-200a01f3b65c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:17.610248 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:17.610384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.610660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.610929 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.611982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:17.613754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:17.714868 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1075/2143] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:17 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:17.717860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:17.736360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae705792-1813-41dc-a789-21bb90df7189 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:17.804626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:17.805127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.805408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.805684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.806915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:17.808434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:17.927582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1069/2144] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:17 2022] POST /identity/v3/users => generated 284 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:17.929107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:17.940176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc14ea4-f2b9-4e05-8d8d-a0211f665063 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:17.944957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:17.948276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.948728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.949139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.950270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:17.958088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:17.965715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1076/2145] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:17.966905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:17.984553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dff2b7-99e4-4a17-ac82-ace3e7400455 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:17.989776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e19199-c761-4c82-8554-bb5a1f403aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:17.990573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19199-c761-4c82-8554-bb5a1f403aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:17.991050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19199-c761-4c82-8554-bb5a1f403aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:17.994643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e19199-c761-4c82-8554-bb5a1f403aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:17.994643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19199-c761-4c82-8554-bb5a1f403aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:17.994643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e19199-c761-4c82-8554-bb5a1f403aa5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:18.011899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1070/2146] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:18.019009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:18.020541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:18.024615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c194-d164-4602-9d54-db84c35cbf1f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:18.030007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:18.030790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:18.031232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:18.031505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] PATH_INFO: `/v3/projects/30545bbfdecf437c999ac0f3143b85b7/users/91ef3d8e3fc54ce6b393f0133f08a902/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:18.049926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30545bbfdecf437c999ac0f3143b85b7, user_id=91ef3d8e3fc54ce6b393f0133f08a902, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:18.052615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:18.158735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1077/2147] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:18 2022] PUT /identity/v3/projects/30545bbfdecf437c999ac0f3143b85b7/users/91ef3d8e3fc54ce6b393f0133f08a902/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:18.170615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:18.170892 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:18.171166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:18.215456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] MFA Rules not processed for user `91ef3d8e3fc54ce6b393f0133f08a902`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:18.216929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:18.296227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1071/2148] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:18.299026 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:18.393567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e99ca-57ff-4397-a0c1-de2648022d41 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:18.398564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:18.403594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:18.403594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:18.403594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:18.403594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:18.403594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:18.523446 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1078/2149] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:18 2022] POST /identity/v3/projects => generated 367 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:18.524277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:18.539812 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0258f-87a4-4a88-9080-4b5b02cadfcb None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:18.590549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:18.612406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:18.613028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:18.613307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:18.613575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:18.614270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:18.616297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:18.720059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1072/2150] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:18 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:18.723405 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:18.734149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d214d0f-938e-49c9-b41e-57d14dcb768f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:18.739037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:18.739644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:18.739925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:18.740232 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:18.740910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:18.742669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:18.753703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1079/2151] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:18.757984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:18.767971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0ea1-457f-4851-834a-625fb1aa00a0 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:18.772631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:18.773308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:18.773642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:18.773868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] PATH_INFO: `/v3/projects/46b1bf33ab86438fbe4cfba074792f72/users/863be18857a64de5821794166c83736d/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:18.789039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b1bf33ab86438fbe4cfba074792f72, user_id=863be18857a64de5821794166c83736d, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:18.792343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:18.904143 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1073/2152] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:18 2022] PUT /identity/v3/projects/46b1bf33ab86438fbe4cfba074792f72/users/863be18857a64de5821794166c83736d/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:18.910256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:18.921592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb35dd-e683-447b-84e0-3c1a769e488d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:19.001417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7a1b-23a6-41bc-abef-98b5155bdb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:19.002034 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7a1b-23a6-41bc-abef-98b5155bdb13 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:19.002319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7a1b-23a6-41bc-abef-98b5155bdb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:19.002608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7a1b-23a6-41bc-abef-98b5155bdb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:19.003401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f7a1b-23a6-41bc-abef-98b5155bdb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:19.005180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f7a1b-23a6-41bc-abef-98b5155bdb13 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:19.019916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1080/2153] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:19.021286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:19.035360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbcffb-5ce7-4625-a201-7366f25c55f9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:19.042540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:19.044163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:19.044938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:19.045952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] PATH_INFO: `/v3/projects/46b1bf33ab86438fbe4cfba074792f72/users/863be18857a64de5821794166c83736d/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:19.049024 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b1bf33ab86438fbe4cfba074792f72, user_id=863be18857a64de5821794166c83736d, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:19.052966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:19.084102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1074/2154] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:19 2022] PUT /identity/v3/projects/46b1bf33ab86438fbe4cfba074792f72/users/863be18857a64de5821794166c83736d/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:19.093239 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6426ec-3305-4306-91e8-602061175246 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:19.093528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6426ec-3305-4306-91e8-602061175246 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:19.093802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6426ec-3305-4306-91e8-602061175246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:19.127827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1c6426ec-3305-4306-91e8-602061175246 None None] MFA Rules not processed for user `863be18857a64de5821794166c83736d`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:19.128883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1c6426ec-3305-4306-91e8-602061175246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:19.176372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:19.234870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1081/2155] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:19.247267 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:19.249241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:19.334178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10eb84f-89c3-41d0-b1fd-2bef46bbf081 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:19.338736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbce60a-2a33-4dbc-8a9c-ba2c24bf493a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:19.339663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce60a-2a33-4dbc-8a9c-ba2c24bf493a service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:19.340578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce60a-2a33-4dbc-8a9c-ba2c24bf493a service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:19.340867 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce60a-2a33-4dbc-8a9c-ba2c24bf493a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:19.352956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbce60a-2a33-4dbc-8a9c-ba2c24bf493a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:19.359397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbce60a-2a33-4dbc-8a9c-ba2c24bf493a service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:19.368181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1075/2156] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:19.689818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6426ec-3305-4306-91e8-602061175246 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:19.701359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6426ec-3305-4306-91e8-602061175246 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:19.705373 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:19.705993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:19.706274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:19.706551 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:19.769144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:19.770585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:19.794319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1082/2157] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:20.959177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:20.959704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:20.959704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:20.996760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:20.998013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:21.053551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1076/2158] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 10 17:14:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:21.066305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:21.067775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:21.141325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176a0df-8e7c-4e54-bd18-649c805156f2 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:21.146340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00079288-2321-44ad-8bca-178ab06ce217 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:21.146959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00079288-2321-44ad-8bca-178ab06ce217 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:21.147219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00079288-2321-44ad-8bca-178ab06ce217 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:21.147488 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00079288-2321-44ad-8bca-178ab06ce217 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:21.155471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00079288-2321-44ad-8bca-178ab06ce217 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:21.157059 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00079288-2321-44ad-8bca-178ab06ce217 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:21.165274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1083/2159] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:14:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:14:26.207227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:26.251363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:26.271699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da6b83-cf2d-44a7-8e4b-ab7119258492 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:26.277545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:26.278374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:26.279120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:26.279433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] PATH_INFO: `/v3/users/91ef3d8e3fc54ce6b393f0133f08a902` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:26.280945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ef3d8e3fc54ce6b393f0133f08a902)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:26.282973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:26.511738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ef3d8e3fc54ce6b393f0133f08a902'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:26.581798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ef3d8e3fc54ce6b393f0133f08a902'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:26.582423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ef3d8e3fc54ce6b393f0133f08a902'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:26.583038 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ef3d8e3fc54ce6b393f0133f08a902'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:26.598452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ef3d8e3fc54ce6b393f0133f08a902'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:26.610802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1077/2160] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:26 2022] DELETE /identity/v3/users/91ef3d8e3fc54ce6b393f0133f08a902 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:26.615529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:26.616015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:26.616144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:26.636464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:26.637673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:26.698469 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1084/2161] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:26.707946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:26.709377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:26.765831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abd9e8-c1ed-4be5-aff5-7fb7de78e8a2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:26.769289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:26.769889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:26.770169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:26.770440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:26.779764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:26.781209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:26.790528 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1078/2162] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:27.176399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:27.177893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:27.188553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:27.192436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34392ad-4db7-45b3-a4ff-f50c71fe7844 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:27.198809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:27.201941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:27.201941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:27.201941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] PATH_INFO: `/v3/projects/30545bbfdecf437c999ac0f3143b85b7` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:27.201941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] RBAC: Authorizing `identity:delete_project(project_id=30545bbfdecf437c999ac0f3143b85b7)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:27.203292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:27.307142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30545bbfdecf437c999ac0f3143b85b7', 'request_id': 'req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.392944 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30545bbfdecf437c999ac0f3143b85b7', 'request_id': 'req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.398823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30545bbfdecf437c999ac0f3143b85b7', 'request_id': 'req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.410189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] The token cache is being invalidate because project 30545bbfdecf437c999ac0f3143b85b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:27.410479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.411116 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.412624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1085/2163] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:27 2022] DELETE /identity/v3/projects/30545bbfdecf437c999ac0f3143b85b7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:27.417302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:27.427132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173e55-df27-4689-ac3e-da8d1124dbf0 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:27.482505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:27.483130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:27.483410 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:27.483685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] PATH_INFO: `/v3/users/863be18857a64de5821794166c83736d` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:27.484830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=863be18857a64de5821794166c83736d)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:27.486718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:27.488958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:27.663564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863be18857a64de5821794166c83736d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.730366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863be18857a64de5821794166c83736d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.730966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863be18857a64de5821794166c83736d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.731583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863be18857a64de5821794166c83736d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.744545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-411adcbf-3519-472a-98ca-be102ba65caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863be18857a64de5821794166c83736d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:27.757396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1079/2164] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:27 2022] DELETE /identity/v3/users/863be18857a64de5821794166c83736d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:28.142705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.144153 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:28.157596 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b83aa-f563-4ca0-b5aa-3c7fe84baefe admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.222708 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.223258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.223538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.223806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] PATH_INFO: `/v3/projects/46b1bf33ab86438fbe4cfba074792f72` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.225053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b1bf33ab86438fbe4cfba074792f72)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.226999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:28.331217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b1bf33ab86438fbe4cfba074792f72', 'request_id': 'req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:28.392924 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b1bf33ab86438fbe4cfba074792f72', 'request_id': 'req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:28.398607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b1bf33ab86438fbe4cfba074792f72', 'request_id': 'req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:28.410094 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] The token cache is being invalidate because project 46b1bf33ab86438fbe4cfba074792f72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:28.410376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:28.411023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4fcda0ab-2e6b-47ac-873a-2874d488bd1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:28.412559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1086/2165] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:28 2022] DELETE /identity/v3/projects/46b1bf33ab86438fbe4cfba074792f72 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:28.417428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.417895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.418011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.438158 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:28.439368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:28.498641 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1080/2166] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:28.507541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.507823 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.508133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.527866 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:28.529166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:28.536948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1087/2167] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:28.540958 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.550284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a3ab5-d9b9-4fd0-aeeb-39d07b237b27 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.554751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.555335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.555612 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.555883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.556547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:28.557473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.559075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:28.570483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1081/2168] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:14:28.575114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.584101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0666cd-40ef-4ce3-a4b0-ce9ddb15829a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.588701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.589278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.589552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.589817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.590436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.592328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:28.601464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:28.669217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1088/2169] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:28 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:28.673251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.684005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e661-b23a-4aca-94e3-e58253974a41 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.738274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.738799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.739073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.739341 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.739966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.741878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:28.816214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:28.822032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1082/2170] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:28 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:28.826308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.836601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995a62b-02b7-479d-88fb-eed86905fe27 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.841308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.841901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.842187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.842459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.843112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.844793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:28.855481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1089/2171] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:28.859437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.870478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a808e8-72d6-4337-9a7e-e0520bdb47db admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.875848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fb8c9-0866-457e-af5b-1f49e77d6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.876474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fb8c9-0866-457e-af5b-1f49e77d6f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.876754 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fb8c9-0866-457e-af5b-1f49e77d6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.877027 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fb8c9-0866-457e-af5b-1f49e77d6f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.877674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fb8c9-0866-457e-af5b-1f49e77d6f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.879250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fb8c9-0866-457e-af5b-1f49e77d6f6c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:28.890505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1083/2172] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:28.894640 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:28.905136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e696b31-cdfd-4966-9c3e-ec5d548a0994 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:28.909978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:28.910673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:28.910954 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:28.911229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] PATH_INFO: `/v3/projects/022d01b632814c03a3f7e1b8ef3fc205/users/dc03584b13ac4375bc52d346bfb8f227/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:28.930994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=022d01b632814c03a3f7e1b8ef3fc205, user_id=dc03584b13ac4375bc52d346bfb8f227, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:28.934298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:29.021999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1090/2173] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:28 2022] PUT /identity/v3/projects/022d01b632814c03a3f7e1b8ef3fc205/users/dc03584b13ac4375bc52d346bfb8f227/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:29.031964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:29.032277 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:29.032550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:29.061395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] MFA Rules not processed for user `dc03584b13ac4375bc52d346bfb8f227`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:29.062588 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:29.131296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1084/2174] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:29.141902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:29.143545 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:29.199139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5c54b-9a8c-438c-b138-7b850e027be9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:29.202775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:29.203366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:29.203642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:29.203915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:29.211976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:29.213908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:29.222449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1091/2175] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:36.029937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:36.033325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:36.120129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d929100-6934-4e28-91bf-69044583ff65 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:36.125530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:36.126199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:36.126482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:36.126756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:36.129730 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:36.151457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:36.165716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:36.165716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:36.167284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:36.169637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1085/2176] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:36.230481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d2283-61bc-4d1a-b4a8-33811130ae2d service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:36.235004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:36.235632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:36.235907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:36.236228 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] PATH_INFO: `/v3/users/1e8d2e4a8a334eea973280b3819219e3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:36.237356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e8d2e4a8a334eea973280b3819219e3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:36.239260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:36.447356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e8d2e4a8a334eea973280b3819219e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.448602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e8d2e4a8a334eea973280b3819219e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.501895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e8d2e4a8a334eea973280b3819219e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.507164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:36.516862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e8d2e4a8a334eea973280b3819219e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.517819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-088fbb1d-d8a0-4680-8f20-34cbf1ebd632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e8d2e4a8a334eea973280b3819219e3'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.530645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1092/2177] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:36 2022] DELETE /identity/v3/users/1e8d2e4a8a334eea973280b3819219e3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:36.760742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:36.772585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41e4ca-9a23-4485-92c1-40a2ce6b3ddb service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:36.836994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:36.837624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:36.837899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:36.838214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] PATH_INFO: `/v3/projects/2488f5601066432a8652433df5702119` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:36.839328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2488f5601066432a8652433df5702119)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:36.841248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:36.849574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:36.972767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2488f5601066432a8652433df5702119', 'request_id': 'req-600f1b82-6a49-41bc-9133-eccfe0464cd7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.978485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2488f5601066432a8652433df5702119', 'request_id': 'req-600f1b82-6a49-41bc-9133-eccfe0464cd7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:36.986822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2488f5601066432a8652433df5702119', 'request_id': 'req-600f1b82-6a49-41bc-9133-eccfe0464cd7'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:37.097061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] The token cache is being invalidate because project 2488f5601066432a8652433df5702119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:37.097697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:37.098544 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-600f1b82-6a49-41bc-9133-eccfe0464cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:37.100223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1086/2178] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:36 2022] DELETE /identity/v3/projects/2488f5601066432a8652433df5702119 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:37.110272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.110614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.110928 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.132558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:37.133763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:37.197253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1093/2179] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:37.206506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.207006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.207497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.230046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:37.231705 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:37.244455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1087/2180] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:37.249383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.259744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d191e3-7558-4a4f-a371-32eba2625b4e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.268250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.268859 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.269131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.269399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.270121 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:37.270419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.272158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:37.287538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1094/2181] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:14:37.289125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.299300 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0920e-b5ff-4ffc-b556-12674ca43215 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.304647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.305273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.305550 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.305820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.306559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.308567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:37.428534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1088/2182] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:37 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:37.433029 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.443195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97776f-d689-44fc-82ba-a9148a9817b4 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.504226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.504824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.505102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.505373 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.506011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.507894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:37.627241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1095/2183] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:37 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:37.631239 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.641503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010cbd9-177f-49eb-8b55-9fed5aafbf48 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.646352 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.646948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.647225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.647497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.648170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.649761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:37.660081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1089/2184] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:37.663962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.674540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92770585-8a33-4fe1-a5c8-1c88558d73f6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.679150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6c32b-34b1-47cf-a1aa-b16455b9f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.679735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6c32b-34b1-47cf-a1aa-b16455b9f90c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.680010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6c32b-34b1-47cf-a1aa-b16455b9f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.680307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6c32b-34b1-47cf-a1aa-b16455b9f90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.680935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6c32b-34b1-47cf-a1aa-b16455b9f90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.682531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6c32b-34b1-47cf-a1aa-b16455b9f90c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:37.692853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1096/2185] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:37.697241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.707372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1c46b-f04a-47b8-8a26-2e2aa59c7994 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.711855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.712620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.712899 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.713776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] PATH_INFO: `/v3/projects/89b036c16efb4407822bf667b6da97f0/users/67f464d60dc54acf9da9fc24ba33b8ee/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.727827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b036c16efb4407822bf667b6da97f0, user_id=67f464d60dc54acf9da9fc24ba33b8ee, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.730513 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:37.815747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1090/2186] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:37 2022] PUT /identity/v3/projects/89b036c16efb4407822bf667b6da97f0/users/67f464d60dc54acf9da9fc24ba33b8ee/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:37.827014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.827323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.827595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.856117 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] MFA Rules not processed for user `67f464d60dc54acf9da9fc24ba33b8ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:37.857320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:37.891693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:37.925442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1097/2187] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:37.930039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:37.991584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aa9b9-c03d-4a6e-b91f-7a351112eba2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:37.996253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:37.996851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:37.997128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:37.997398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:37.998029 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:37.999922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.086692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1091/2188] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:37 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:38.093166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:38.104095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382a2ca-3ce2-4555-9cce-a0c4c738e7b0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:38.160942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:38.161471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.161750 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.162023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.162664 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:38.164632 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.263187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1098/2189] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:38 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:38.267109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:38.280772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa506a0-3122-4ee6-960c-9dd73ebbfcc5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:38.285714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:38.286320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.286706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.286864 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.287574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:38.289150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.300480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1092/2190] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:38.304763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:38.316502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30c6bc-99fc-4156-8bbe-06333adaba14 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:38.322376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:38.322376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.323189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.323386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] PATH_INFO: `/v3/projects/173bff14d53a429a9ff7cbd21b46afdf/users/9b0a897e839540cea5207d3371037367/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.342250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173bff14d53a429a9ff7cbd21b46afdf, user_id=9b0a897e839540cea5207d3371037367, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:38.344907 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.404602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1099/2191] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:38 2022] PUT /identity/v3/projects/173bff14d53a429a9ff7cbd21b46afdf/users/9b0a897e839540cea5207d3371037367/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:38.408854 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:38.419542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea34b4-eb36-4405-be93-df1047b5b02d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:38.423071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:38.475665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877bdca-e693-4f0d-acb5-9b82c686896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:38.476290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877bdca-e693-4f0d-acb5-9b82c686896b admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.476572 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877bdca-e693-4f0d-acb5-9b82c686896b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.476841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877bdca-e693-4f0d-acb5-9b82c686896b admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.477471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877bdca-e693-4f0d-acb5-9b82c686896b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:38.479551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877bdca-e693-4f0d-acb5-9b82c686896b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.489628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1093/2192] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:38.493719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:38.504478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936f23f-5278-4c97-9ce5-aed4d0acb960 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:38.509069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:38.510395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.510395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.510395 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] PATH_INFO: `/v3/projects/173bff14d53a429a9ff7cbd21b46afdf/users/9b0a897e839540cea5207d3371037367/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.512162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173bff14d53a429a9ff7cbd21b46afdf, user_id=9b0a897e839540cea5207d3371037367, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:38.514739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.525398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1100/2193] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:38 2022] PUT /identity/v3/projects/173bff14d53a429a9ff7cbd21b46afdf/users/9b0a897e839540cea5207d3371037367/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:38.535955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.536279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.536563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.565184 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] MFA Rules not processed for user `9b0a897e839540cea5207d3371037367`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:38.566391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:38.634235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1094/2194] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:38.645464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:38.647130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:38.704097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456119c-2799-423c-9acf-2b234ef97fa8 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:38.707636 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:38.708307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:38.708585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:38.708849 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:38.717567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:38.718975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:38.728197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1101/2195] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:39.362480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:39.363918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:39.427711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d15fa-f08b-4695-9b45-68a0e25fa8f0 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:39.432297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:39.432922 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:39.433197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:39.433465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] PATH_INFO: `/v3/users/dc03584b13ac4375bc52d346bfb8f227` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:39.434562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc03584b13ac4375bc52d346bfb8f227)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:39.436495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:39.609953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:39.621051 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0309cb5-3ccf-4cba-a36a-cb1ebe7db369 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:39.625634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:39.626254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:39.626532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:39.626803 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] PATH_INFO: `/v3/users/67f464d60dc54acf9da9fc24ba33b8ee` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:39.627905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f464d60dc54acf9da9fc24ba33b8ee)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:39.629818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:39.639522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:39.652150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:39.654336 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc03584b13ac4375bc52d346bfb8f227'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.701204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc03584b13ac4375bc52d346bfb8f227'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.701831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc03584b13ac4375bc52d346bfb8f227'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.702448 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc03584b13ac4375bc52d346bfb8f227'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.714596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-633cbae9-6fa6-42ba-8de4-dc92550a64c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc03584b13ac4375bc52d346bfb8f227'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.726647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1095/2196] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:39 2022] DELETE /identity/v3/users/dc03584b13ac4375bc52d346bfb8f227 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:39.731059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:39.731345 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:39.731618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:39.750505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:39.751702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:39.767044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f464d60dc54acf9da9fc24ba33b8ee'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.767722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f464d60dc54acf9da9fc24ba33b8ee'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.811120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1096/2197] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:39.821018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:39.822318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:39.837914 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f464d60dc54acf9da9fc24ba33b8ee'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.851213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f464d60dc54acf9da9fc24ba33b8ee'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.851901 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a914c031-e7be-4656-8c0c-ce86db9546e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f464d60dc54acf9da9fc24ba33b8ee'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:39.864164 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1102/2198] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:39 2022] DELETE /identity/v3/users/67f464d60dc54acf9da9fc24ba33b8ee => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:39.868997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:39.869896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:39.870285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:39.883885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bed8da-d486-480a-baf5-eee6dd8a2f29 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:39.891876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:39.893113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:39.932913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:39.933523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:39.933800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:39.934175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:39.951261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:39.956807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1103/2199] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:39.966943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:39.976888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046df3f-10ed-41c7-9f94-ad1623c514d7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:39.981369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:39.982333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:39.982459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:39.982764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:39.993459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:39.995726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:39.999032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:40.000500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:40.006597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1104/2200] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:40.010128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1097/2201] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:40.513022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:40.513022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:40.523609 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e4bbe-92ee-4fb8-b992-e757bd292968 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:40.528788 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:40.529432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.529714 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.529988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] PATH_INFO: `/v3/projects/89b036c16efb4407822bf667b6da97f0` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.534158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b036c16efb4407822bf667b6da97f0)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:40.534158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:40.547486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:40.549006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:40.564918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4bdd6-f836-4ab4-a063-dc9bbcb55962 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:40.568827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:40.569453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.569761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.570314 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] PATH_INFO: `/v3/projects/022d01b632814c03a3f7e1b8ef3fc205` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.571447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=022d01b632814c03a3f7e1b8ef3fc205)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:40.573674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:40.645002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b036c16efb4407822bf667b6da97f0', 'request_id': 'req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.673878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022d01b632814c03a3f7e1b8ef3fc205', 'request_id': 'req-428bc908-0953-4adf-a96e-35269d80c214'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.679307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022d01b632814c03a3f7e1b8ef3fc205', 'request_id': 'req-428bc908-0953-4adf-a96e-35269d80c214'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.685800 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022d01b632814c03a3f7e1b8ef3fc205', 'request_id': 'req-428bc908-0953-4adf-a96e-35269d80c214'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.713957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b036c16efb4407822bf667b6da97f0', 'request_id': 'req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.719917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b036c16efb4407822bf667b6da97f0', 'request_id': 'req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.731319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] The token cache is being invalidate because project 89b036c16efb4407822bf667b6da97f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:40.731611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.732305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.734071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1105/2202] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:40 2022] DELETE /identity/v3/projects/89b036c16efb4407822bf667b6da97f0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:40.737876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:40.739299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:40.749500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:40.781853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] The token cache is being invalidate because project 022d01b632814c03a3f7e1b8ef3fc205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:40.782142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.782814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-428bc908-0953-4adf-a96e-35269d80c214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:40.784562 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1098/2203] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:40 2022] DELETE /identity/v3/projects/022d01b632814c03a3f7e1b8ef3fc205 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:40.789057 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff647919-a3c0-400c-9030-cf8b27c1bd0c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.789346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff647919-a3c0-400c-9030-cf8b27c1bd0c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.789621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff647919-a3c0-400c-9030-cf8b27c1bd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.804421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c2ebe-3d0d-4b18-9be9-33c7496c7ea0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:40.810906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ff647919-a3c0-400c-9030-cf8b27c1bd0c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:40.810906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ff647919-a3c0-400c-9030-cf8b27c1bd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:40.862701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:40.863337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.863619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.863902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] PATH_INFO: `/v3/users/9b0a897e839540cea5207d3371037367` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.865092 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0a897e839540cea5207d3371037367)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:40.867302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:40.876113 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1099/2204] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:40.885636 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.885926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.886204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.906222 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:40.907414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:40.915388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1100/2205] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:40.919516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:40.928358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bb499-e5e4-4b62-a206-ab9bf48c54ba None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:40.933019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:40.933592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.933873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.934147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.934780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:40.935083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:40.936755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:40.949053 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1101/2206] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:14:40.953139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:40.962599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d229d7c-b835-4e3e-9b0f-a48e2ce9578b admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:40.967817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:40.969169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:40.969465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:40.969742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:40.970429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:40.972376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:40.988291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:41.041177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0a897e839540cea5207d3371037367'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.041845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0a897e839540cea5207d3371037367'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.058036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1102/2207] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:40 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:41.062019 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.063599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:41.090095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0a897e839540cea5207d3371037367'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.102618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0a897e839540cea5207d3371037367'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.103416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0a897e839540cea5207d3371037367'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.115810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1106/2208] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:40 2022] DELETE /identity/v3/users/9b0a897e839540cea5207d3371037367 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:41.131162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc57a4-f037-4f3f-9d42-7d3984ef94f9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.190438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.190980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.191262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.191534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.192205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.194303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.250995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1103/2209] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:41 2022] POST /identity/v3/users => generated 281 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:41.255027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.265464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fa6a4-7434-431a-9b73-f1b977cab8ce admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.270543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.271468 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.271468 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.271739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.272550 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.274031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.284686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1107/2210] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:41.289188 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.299805 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30479a1-2638-4eb3-9b57-07f54d330bf4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.304496 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d64e-faf7-47b5-a196-af58d2048cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.305090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d64e-faf7-47b5-a196-af58d2048cde admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.305364 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d64e-faf7-47b5-a196-af58d2048cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.305752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d64e-faf7-47b5-a196-af58d2048cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.306296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d64e-faf7-47b5-a196-af58d2048cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.308446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d64e-faf7-47b5-a196-af58d2048cde admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.329449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1104/2211] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:41.333054 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.343181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d180-34c2-422c-8499-6f7b6653fe00 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.347829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.348520 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.348798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.349068 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] PATH_INFO: `/v3/projects/c9f25b93211e4eb696be4afdf32f1457/users/6dcbd50953cb4a53b0bf2bf3a13f5b2e/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.364423 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f25b93211e4eb696be4afdf32f1457, user_id=6dcbd50953cb4a53b0bf2bf3a13f5b2e, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.367084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.428651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:41.434033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1108/2212] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:41 2022] PUT /identity/v3/projects/c9f25b93211e4eb696be4afdf32f1457/users/6dcbd50953cb4a53b0bf2bf3a13f5b2e/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:41.444270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.444522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.444819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.472905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] MFA Rules not processed for user `6dcbd50953cb4a53b0bf2bf3a13f5b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:41.474138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:41.537947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1105/2213] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:41.549436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.551194 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:41.576607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.578709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:41.592779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:41.632329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e081cfc-a182-4f5f-a3ac-99051a2cff67 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.637029 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.637660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.637942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.638074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57abdbd-4bb5-438e-8167-dde20691fffe None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.638218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] PATH_INFO: `/v3/projects/173bff14d53a429a9ff7cbd21b46afdf` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.639385 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=173bff14d53a429a9ff7cbd21b46afdf)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.641598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.641884 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.642192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.642471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.642744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.652302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.653851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.662644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1106/2214] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:41.679417 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:41.690187 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b49b2-8b07-4851-805a-c9a7b25308cf service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:41.694721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:41.695521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.695794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.696090 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] PATH_INFO: `/v3/users/6dcbd50953cb4a53b0bf2bf3a13f5b2e` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.697198 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcbd50953cb4a53b0bf2bf3a13f5b2e)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:41.699154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:41.748157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173bff14d53a429a9ff7cbd21b46afdf', 'request_id': 'req-3f2d5aad-5870-4030-8b07-47e638755bea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.844405 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173bff14d53a429a9ff7cbd21b46afdf', 'request_id': 'req-3f2d5aad-5870-4030-8b07-47e638755bea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.850231 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173bff14d53a429a9ff7cbd21b46afdf', 'request_id': 'req-3f2d5aad-5870-4030-8b07-47e638755bea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.862796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] The token cache is being invalidate because project 173bff14d53a429a9ff7cbd21b46afdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:41.863087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.863760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-3f2d5aad-5870-4030-8b07-47e638755bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.865430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1109/2215] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:41 2022] DELETE /identity/v3/projects/173bff14d53a429a9ff7cbd21b46afdf => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:41.873304 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d78cd-f568-4248-bc48-9c33763c4738 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.873593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d78cd-f568-4248-bc48-9c33763c4738 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.873870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d78cd-f568-4248-bc48-9c33763c4738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.887545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcbd50953cb4a53b0bf2bf3a13f5b2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.894020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-b44d78cd-f568-4248-bc48-9c33763c4738 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:41.895141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b44d78cd-f568-4248-bc48-9c33763c4738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:41.941591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcbd50953cb4a53b0bf2bf3a13f5b2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.942389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcbd50953cb4a53b0bf2bf3a13f5b2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.943013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcbd50953cb4a53b0bf2bf3a13f5b2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.954557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1110/2216] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:41.962130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1ce3a444-9410-4d9f-8618-63b82a6a4e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcbd50953cb4a53b0bf2bf3a13f5b2e'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:41.964112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.964397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.964962 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.975508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1107/2217] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:41 2022] DELETE /identity/v3/users/6dcbd50953cb4a53b0bf2bf3a13f5b2e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:41.980305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:41.980585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:41.980858 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:41.985230 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:41.986453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:41.999185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:42.000389 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:42.013624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:42.048991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1111/2218] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:42.053019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.061261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1108/2219] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:42.064436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9f9ba-8452-4619-9881-9e036ef87801 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.068424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.069071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.069349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.069615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.070517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:42.070816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.071254 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.072517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.072807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:42.089103 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1112/2220] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:14:42.090099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.099986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763bf5-402b-4feb-a77f-4b150fb6d216 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.105220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.105799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.106077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.106344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.106956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.108806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:42.108947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.135365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7cac5-b5b9-4dd5-9e74-45771138cf94 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.139011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.139651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.139975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.143232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.149114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.151045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.160354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1109/2221] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:42.198802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1113/2222] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:42 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:42.200141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.209970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08f14c-b8f3-4996-9085-7a59cf3ee4e7 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.267861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.268498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.268785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.269055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.269676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.271644 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.390832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1110/2223] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:42 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:42.394829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.406592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d90a50-c2d8-4e8b-a38d-f12200fc259c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.411225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.411898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.412290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.412985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.413409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.416822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.425574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1114/2224] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:42.429672 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.439726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456e920-0e1c-4a0e-8f52-1a9a1692e458 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.444783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.445784 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.446093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.446480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.447151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.449457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.459884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1111/2225] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:42.469057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.517724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224d37c-8a61-456a-b956-f8643e3d7883 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.522346 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76ed91-6a55-430f-852a-28601e56fd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.523150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76ed91-6a55-430f-852a-28601e56fd44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.523441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76ed91-6a55-430f-852a-28601e56fd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.523695 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76ed91-6a55-430f-852a-28601e56fd44 admin admin] PATH_INFO: `/v3/projects/1fb8d1901b3d4e8c93fe2921261aff3d/users/cf4fea38f6a042cb860ed0d3b87d1903/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.538487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76ed91-6a55-430f-852a-28601e56fd44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb8d1901b3d4e8c93fe2921261aff3d, user_id=cf4fea38f6a042cb860ed0d3b87d1903, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.541307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76ed91-6a55-430f-852a-28601e56fd44 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.608382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.608382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:42.662277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1115/2226] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:42 2022] PUT /identity/v3/projects/1fb8d1901b3d4e8c93fe2921261aff3d/users/cf4fea38f6a042cb860ed0d3b87d1903/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:42.666101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c680489-b8ad-43d5-bae9-e9fc60642456 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.672526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.672817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.673093 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.707129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] MFA Rules not processed for user `cf4fea38f6a042cb860ed0d3b87d1903`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:42.708584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:42.746412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:42.746412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.746412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.746412 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.747214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] PATH_INFO: `/v3/projects/c9f25b93211e4eb696be4afdf32f1457` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.747214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9f25b93211e4eb696be4afdf32f1457)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.747214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.786612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1116/2227] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:42.792592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:42.803453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a73e3-b07d-4337-a951-bd340bf2f0b3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:42.815412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:42.815412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:42.815412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:42.815412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:42.815830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:42.815830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:42.821678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:42.890976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9f25b93211e4eb696be4afdf32f1457', 'request_id': 'req-10e4ab08-a579-41c3-8367-6e335c93f88a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:42.896480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9f25b93211e4eb696be4afdf32f1457', 'request_id': 'req-10e4ab08-a579-41c3-8367-6e335c93f88a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:42.903430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9f25b93211e4eb696be4afdf32f1457', 'request_id': 'req-10e4ab08-a579-41c3-8367-6e335c93f88a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:42.938247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1117/2228] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:42 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:42.942123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.005625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcacc8-b8f7-4e29-8e0c-e4acd3d98ea9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.010407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.011011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.011288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.011558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.011869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] The token cache is being invalidate because project c9f25b93211e4eb696be4afdf32f1457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:43.012144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:43.012259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.012777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-10e4ab08-a579-41c3-8367-6e335c93f88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:43.014157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.014279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1112/2229] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:42 2022] DELETE /identity/v3/projects/c9f25b93211e4eb696be4afdf32f1457 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:43.019096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40282e74-d64c-4292-a31d-ad794b69792c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.019426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40282e74-d64c-4292-a31d-ad794b69792c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.019770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40282e74-d64c-4292-a31d-ad794b69792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.079811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-40282e74-d64c-4292-a31d-ad794b69792c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:43.081058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-40282e74-d64c-4292-a31d-ad794b69792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.087782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1118/2230] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:42 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:43.091317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.091444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1113/2231] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:43.091535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.096603 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.096884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.097156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.102767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979eac6-81b1-4efe-aa0f-23c2bc9a5f10 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.108888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.109496 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.109778 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.110053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.110692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.112321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.116732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:43.118039 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.122842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1119/2232] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:43.127006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.128416 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1114/2233] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:43.132288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.136987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14297471-d7ee-40fc-ac42-4064f15d25a5 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.145827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.146494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.146768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.147037 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] PATH_INFO: `/v3/projects/412cc1ed7e064987a13b389f560cba09/users/b36bdcb1950146fabb7f86d3288ce2d4/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.149224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b949d-fbbc-4ae3-891e-15d44f7d3c20 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.154437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.155048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.155330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.155606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.156288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:43.156596 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.158235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.163378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412cc1ed7e064987a13b389f560cba09, user_id=b36bdcb1950146fabb7f86d3288ce2d4, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.166032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.170452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1115/2234] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:14:43.174811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.184827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c3e1-0643-4cad-a4eb-089e0b237df5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.189477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.190045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.190332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.190606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.191217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.193102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.228163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1120/2235] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:43 2022] PUT /identity/v3/projects/412cc1ed7e064987a13b389f560cba09/users/b36bdcb1950146fabb7f86d3288ce2d4/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:43.232445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.292108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dd092-f64a-43dd-bb77-13d5be65bef0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.296693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.297282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.297557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.297833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.298486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.300065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.310277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1121/2236] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:43.314137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.323552 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75343dd-4443-4801-8631-4631f5e839b9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.328099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f3a04-2a1e-4d25-b26f-cd9223033f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.328705 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3a04-2a1e-4d25-b26f-cd9223033f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.328987 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3a04-2a1e-4d25-b26f-cd9223033f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.329258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3a04-2a1e-4d25-b26f-cd9223033f84 admin admin] PATH_INFO: `/v3/projects/412cc1ed7e064987a13b389f560cba09/users/b36bdcb1950146fabb7f86d3288ce2d4/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.331134 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f3a04-2a1e-4d25-b26f-cd9223033f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412cc1ed7e064987a13b389f560cba09, user_id=b36bdcb1950146fabb7f86d3288ce2d4, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.331885 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1116/2237] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:43.333735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f3a04-2a1e-4d25-b26f-cd9223033f84 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.335905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.344915 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1122/2238] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:43 2022] PUT /identity/v3/projects/412cc1ed7e064987a13b389f560cba09/users/b36bdcb1950146fabb7f86d3288ce2d4/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:43.356350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.356637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.356911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.384691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] MFA Rules not processed for user `b36bdcb1950146fabb7f86d3288ce2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:43.385940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.391408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:43.404221 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e3d64-d14a-46df-a47c-28f8e49816c2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.412963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:43.455375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1123/2239] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:43.465905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.466521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.466797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.467068 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.467177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.467698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.468641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.469617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.562152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1117/2240] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:43.563150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.565167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15d7a9-e0ca-45e7-a442-bee92059b5be None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.568869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.578477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.578477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.578477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.578798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf1691-8f12-4f91-b5bd-1246e4803a5a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.578798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.579404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.579528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.579683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.579953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.580724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.580931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.582291 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.590707 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1124/2241] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:43.593147 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1118/2242] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:43.596684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.607676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b0b3-fa1c-4954-a448-061c7a819d21 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.612337 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.613071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.613349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.613648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.614275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.615857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.625735 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1125/2243] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:43.629847 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.639934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49d838-482a-4952-8586-cccd8ec0b9b8 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.644631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f127f-0e4c-4d68-9f28-ca8babef800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.645328 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f127f-0e4c-4d68-9f28-ca8babef800d admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.645587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f127f-0e4c-4d68-9f28-ca8babef800d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.645857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f127f-0e4c-4d68-9f28-ca8babef800d admin admin] PATH_INFO: `/v3/projects/96500148c45648c08c070831a057eead/users/0f45191d87854d1f84d190ac9123d4aa/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.661169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f127f-0e4c-4d68-9f28-ca8babef800d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96500148c45648c08c070831a057eead, user_id=0f45191d87854d1f84d190ac9123d4aa, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.663823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f127f-0e4c-4d68-9f28-ca8babef800d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.721655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.723426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.735712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29afb-a505-4fdc-8a2e-1666c9b731f2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.739541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.740285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.740582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.745010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.745010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.770375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1119/2244] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:43 2022] PUT /identity/v3/projects/96500148c45648c08c070831a057eead/users/0f45191d87854d1f84d190ac9123d4aa/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:43.782565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.782851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.783124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.805470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.807003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:43.810758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:43.811026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] MFA Rules not processed for user `0f45191d87854d1f84d190ac9123d4aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:43.812342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:43.817002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1126/2245] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:43.882810 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1120/2246] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:43.888169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:43.899044 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8eb56-16c0-4a56-9b96-673b66e4ffc5 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:43.959773 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:43.960420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:43.960696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:43.960965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:43.961649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:43.964192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.046005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1127/2247] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:43 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:44.050138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:44.061541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31321fd-c66e-44b5-8ed2-597568308f88 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:44.111780 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:44.117769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:44.122135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.122135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.122135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.122135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:44.122135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.206237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1121/2248] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:44 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:44.209168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:44.219454 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859b029-598b-4060-8030-d168c2ee0446 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:44.224604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:44.225241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.225470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.225746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.226384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:44.227966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.238620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1128/2249] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:44.242770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:44.253281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6095a9-0c3c-441d-ac41-bb3e158d4bdf admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:44.257976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:44.258625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.258898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.259172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] PATH_INFO: `/v3/projects/0873014674bc4487b4a4a78e34d8be9a/users/1789afcaa88145a1af58ecbd444ace73/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.274702 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0873014674bc4487b4a4a78e34d8be9a, user_id=1789afcaa88145a1af58ecbd444ace73, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:44.277347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.387629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1122/2250] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:44 2022] PUT /identity/v3/projects/0873014674bc4487b4a4a78e34d8be9a/users/1789afcaa88145a1af58ecbd444ace73/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:44.391494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:44.401310 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5be2a-6c6b-4b69-a16b-3a285cbc00b4 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:44.461104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ef90c-0953-463e-8191-98aace28778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:44.461719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef90c-0953-463e-8191-98aace28778e admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.461999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef90c-0953-463e-8191-98aace28778e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.462274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef90c-0953-463e-8191-98aace28778e admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.462912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef90c-0953-463e-8191-98aace28778e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:44.464537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef90c-0953-463e-8191-98aace28778e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.477806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1129/2251] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:44.479512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:44.493689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f6d7e-6e83-41e8-8f9c-157b6b1672d2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:44.499404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:44.500320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.500600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.500871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] PATH_INFO: `/v3/projects/0873014674bc4487b4a4a78e34d8be9a/users/1789afcaa88145a1af58ecbd444ace73/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.502787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0873014674bc4487b4a4a78e34d8be9a, user_id=1789afcaa88145a1af58ecbd444ace73, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:44.505488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.516445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1123/2252] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:44 2022] PUT /identity/v3/projects/0873014674bc4487b4a4a78e34d8be9a/users/1789afcaa88145a1af58ecbd444ace73/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:44.529111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.529111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.529111 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.558746 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] MFA Rules not processed for user `1789afcaa88145a1af58ecbd444ace73`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:44.559977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:44.617399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:44.627585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1130/2253] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:44.641632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:44.695788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffcec6-f40a-46d8-acdc-ebb2be30369f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:44.699511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4eacf-26b5-46c0-975d-46cff1ddc538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:44.700152 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4eacf-26b5-46c0-975d-46cff1ddc538 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:44.700433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4eacf-26b5-46c0-975d-46cff1ddc538 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:44.700706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4eacf-26b5-46c0-975d-46cff1ddc538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:44.756213 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4eacf-26b5-46c0-975d-46cff1ddc538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:44.757632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4eacf-26b5-46c0-975d-46cff1ddc538 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:44.766380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1124/2254] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:47.770458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:47.848076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c4291-5da6-4b97-8bed-b6d07de6001c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:47.852668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:47.853291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:47.853565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:47.853835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] PATH_INFO: `/v3/users/0f45191d87854d1f84d190ac9123d4aa` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:47.854976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f45191d87854d1f84d190ac9123d4aa)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:47.856913 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:48.025290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f45191d87854d1f84d190ac9123d4aa'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:48.026203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f45191d87854d1f84d190ac9123d4aa'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:48.095826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f45191d87854d1f84d190ac9123d4aa'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:48.110234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f45191d87854d1f84d190ac9123d4aa'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:48.110984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f45191d87854d1f84d190ac9123d4aa'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:48.123658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1131/2255] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:47 2022] DELETE /identity/v3/users/0f45191d87854d1f84d190ac9123d4aa => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:48.128176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:48.128706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:48.128827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:48.147614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:48.148949 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:48.209276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1125/2256] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:48.219010 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:48.220536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:48.303190 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60832d-c9ab-4008-b2f9-09ed42fb9611 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:48.306820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:48.309435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:48.309435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:48.309435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:48.315546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:48.317923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:48.319387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:48.329204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1132/2257] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:48.732975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:48.734362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:48.742675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:48.746368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183933c-bbcf-4903-a109-f69aa2494d59 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:48.751323 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:48.751972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:48.752274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:48.752545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] PATH_INFO: `/v3/users/cf4fea38f6a042cb860ed0d3b87d1903` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:48.753647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4fea38f6a042cb860ed0d3b87d1903)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:48.755574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:48.788600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:48.790386 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:48.802994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd681e-c806-4bfc-84d5-534a8330e296 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:48.807581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:48.808236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:48.808564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:48.808889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] PATH_INFO: `/v3/projects/96500148c45648c08c070831a057eead` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:48.810447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96500148c45648c08c070831a057eead)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:48.812407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:48.978615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4fea38f6a042cb860ed0d3b87d1903'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:48.979745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96500148c45648c08c070831a057eead', 'request_id': 'req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.055201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96500148c45648c08c070831a057eead', 'request_id': 'req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.062624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96500148c45648c08c070831a057eead', 'request_id': 'req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.074146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] The token cache is being invalidate because project 96500148c45648c08c070831a057eead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:49.074439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.075079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.076623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1133/2258] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:48 2022] DELETE /identity/v3/projects/96500148c45648c08c070831a057eead => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:49.080398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:49.081693 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:49.118934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4fea38f6a042cb860ed0d3b87d1903'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.119560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4fea38f6a042cb860ed0d3b87d1903'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.120199 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4fea38f6a042cb860ed0d3b87d1903'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.133897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-0aa6d45e-69b4-4a37-b75a-45182d96411f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4fea38f6a042cb860ed0d3b87d1903'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.146535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb7bc6-5e43-4d4a-b0ac-ddb60c18c092 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:49.151289 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:49.152101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:49.152383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:49.152655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] PATH_INFO: `/v3/users/1789afcaa88145a1af58ecbd444ace73` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:49.153802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1789afcaa88145a1af58ecbd444ace73)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:49.154133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1126/2259] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:48 2022] DELETE /identity/v3/users/cf4fea38f6a042cb860ed0d3b87d1903 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:49.155784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:49.158714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:49.159127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:49.159493 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:49.179173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:49.180399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:49.238134 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1127/2260] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:49.247836 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:49.249264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:49.304365 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30532fe-8af8-47a8-aebc-e64d5dd85afd None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:49.307956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:49.308578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:49.308853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:49.309121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:49.318050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:49.319394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:49.329159 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1128/2261] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:49.338567 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1789afcaa88145a1af58ecbd444ace73'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.340919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1789afcaa88145a1af58ecbd444ace73'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.393427 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1789afcaa88145a1af58ecbd444ace73'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.407026 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1789afcaa88145a1af58ecbd444ace73'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.407734 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1789afcaa88145a1af58ecbd444ace73'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.415692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:49.421415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1134/2262] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:49 2022] DELETE /identity/v3/users/1789afcaa88145a1af58ecbd444ace73 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:49.744564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:49.746079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:49.758986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f36f1-8049-40fd-8e3a-970a3cf5ed4c service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:49.777741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:49.817770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:49.818439 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:49.818684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:49.819071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] PATH_INFO: `/v3/projects/1fb8d1901b3d4e8c93fe2921261aff3d` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:49.820125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb8d1901b3d4e8c93fe2921261aff3d)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:49.822298 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:49.824006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:49.826200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:49.840151 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcab87e-9dbd-4c2e-ba87-987515401409 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:49.846414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:49.846888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:49.847240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:49.847696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] PATH_INFO: `/v3/projects/0873014674bc4487b4a4a78e34d8be9a` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:49.849087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0873014674bc4487b4a4a78e34d8be9a)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:49.851060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:49.913720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb8d1901b3d4e8c93fe2921261aff3d', 'request_id': 'req-719df667-bdf2-4c2b-9bcb-d1b986b400ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.919108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb8d1901b3d4e8c93fe2921261aff3d', 'request_id': 'req-719df667-bdf2-4c2b-9bcb-d1b986b400ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.925397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb8d1901b3d4e8c93fe2921261aff3d', 'request_id': 'req-719df667-bdf2-4c2b-9bcb-d1b986b400ff'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.941942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0873014674bc4487b4a4a78e34d8be9a', 'request_id': 'req-9405ace0-b98f-464f-bc06-655e634cf58d'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.992435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] The token cache is being invalidate because project 1fb8d1901b3d4e8c93fe2921261aff3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:49.992722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.993332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:49.994833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1129/2263] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:49 2022] DELETE /identity/v3/projects/1fb8d1901b3d4e8c93fe2921261aff3d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:49.999444 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.000785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.027175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0873014674bc4487b4a4a78e34d8be9a', 'request_id': 'req-9405ace0-b98f-464f-bc06-655e634cf58d'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.033294 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0873014674bc4487b4a4a78e34d8be9a', 'request_id': 'req-9405ace0-b98f-464f-bc06-655e634cf58d'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.045053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] The token cache is being invalidate because project 0873014674bc4487b4a4a78e34d8be9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:50.046728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.046728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-9405ace0-b98f-464f-bc06-655e634cf58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.047721 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1135/2264] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:49 2022] DELETE /identity/v3/projects/0873014674bc4487b4a4a78e34d8be9a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:50.052177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19718-01d7-403d-8377-8dc813b57875 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.052382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19718-01d7-403d-8377-8dc813b57875 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.052661 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19718-01d7-403d-8377-8dc813b57875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.064688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df667-bdf2-4c2b-9bcb-d1b986b400ff admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.072840 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e2d19718-01d7-403d-8377-8dc813b57875 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:50.074046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e2d19718-01d7-403d-8377-8dc813b57875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.121223 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.121826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.122104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.122963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] PATH_INFO: `/v3/users/b36bdcb1950146fabb7f86d3288ce2d4` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.124229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36bdcb1950146fabb7f86d3288ce2d4)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.126125 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.135489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1136/2265] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:50.143864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.144195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.144470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.163747 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:50.164980 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.173061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1137/2266] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:50.177088 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.186576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523af6b4-a526-467d-b7c9-532d788c95e7 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.191175 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.191755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.192057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.192333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.192973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:50.193276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.194857 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.199088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:50.208352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1138/2267] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:14:50.212516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.222015 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b560a-100d-48d1-b045-f6ba1313f199 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.226816 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.227384 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.227754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.228065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.228672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.230629 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.259034 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36bdcb1950146fabb7f86d3288ce2d4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.308800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1139/2268] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:50 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:50.313091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.319638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36bdcb1950146fabb7f86d3288ce2d4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.320770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36bdcb1950146fabb7f86d3288ce2d4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.321443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36bdcb1950146fabb7f86d3288ce2d4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.337176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36bdcb1950146fabb7f86d3288ce2d4'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.350172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1130/2269] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:49 2022] DELETE /identity/v3/users/b36bdcb1950146fabb7f86d3288ce2d4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:50.362471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:50.378448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165cc90-4ed3-4801-8136-013f7bab0e8d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.434607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.435768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.436079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.436350 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.436975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.438917 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.524787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1140/2270] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:50 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:50.528894 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.531080 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.544151 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713a5a1-d10b-4c50-ba89-d39ed479f84f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.549655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.550405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.550687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.550961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.551640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.553268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.569108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1131/2271] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:50.572178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.582324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085edf88-4a37-492c-8dd7-22819f15b031 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.588347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946962ea-7877-4d3c-a7af-d2db350e5762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.588948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946962ea-7877-4d3c-a7af-d2db350e5762 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.589225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946962ea-7877-4d3c-a7af-d2db350e5762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.589495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946962ea-7877-4d3c-a7af-d2db350e5762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.590135 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946962ea-7877-4d3c-a7af-d2db350e5762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.591752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946962ea-7877-4d3c-a7af-d2db350e5762 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.601801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1141/2272] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:50.605982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.616638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911f032-fda7-4f3c-999f-5f780d38a618 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.621286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.621929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.622207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.622497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] PATH_INFO: `/v3/projects/a9ddc2babd074a589af9378974450895/users/05f5ee18b63c47cfa7ce789bd004f3b9/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.639796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ddc2babd074a589af9378974450895, user_id=05f5ee18b63c47cfa7ce789bd004f3b9, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.642457 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.701825 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1132/2273] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:50 2022] PUT /identity/v3/projects/a9ddc2babd074a589af9378974450895/users/05f5ee18b63c47cfa7ce789bd004f3b9/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:50.713192 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.718156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.718156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.742502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] MFA Rules not processed for user `05f5ee18b63c47cfa7ce789bd004f3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:50.743749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.754398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.756109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.817766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1142/2274] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:50.828947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.829443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707946-f239-425e-8084-88edc7e9cd08 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.830353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:50.833938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.834561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.834850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.835123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] PATH_INFO: `/v3/projects/412cc1ed7e064987a13b389f560cba09` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.836286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=412cc1ed7e064987a13b389f560cba09)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.838167 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.889623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d7194-5f97-45a4-93ac-af31245d1607 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.893197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:50.893801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:50.894078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:50.894349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:50.902789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:50.904768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:50.908249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:50.914137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1143/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:50.939475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:50.950777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab1a62-8035-46c8-9c8e-3b125605046b service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:50.962194 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:50.967156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412cc1ed7e064987a13b389f560cba09', 'request_id': 'req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.972564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412cc1ed7e064987a13b389f560cba09', 'request_id': 'req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:50.978744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412cc1ed7e064987a13b389f560cba09', 'request_id': 'req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.011495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.012160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.012444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.012719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] PATH_INFO: `/v3/users/05f5ee18b63c47cfa7ce789bd004f3b9` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.014406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f5ee18b63c47cfa7ce789bd004f3b9)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.016352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.043673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] The token cache is being invalidate because project 412cc1ed7e064987a13b389f560cba09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:51.043968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.044651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9584c6a-8510-489c-b0d8-9b1beb26ae9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.046154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1133/2276] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:50 2022] DELETE /identity/v3/projects/412cc1ed7e064987a13b389f560cba09 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:51.050988 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dedc7-bea2-45dd-84d9-c7dd18acfa27 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.051269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dedc7-bea2-45dd-84d9-c7dd18acfa27 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.051538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dedc7-bea2-45dd-84d9-c7dd18acfa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.071520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-266dedc7-bea2-45dd-84d9-c7dd18acfa27 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:51.080344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-266dedc7-bea2-45dd-84d9-c7dd18acfa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.081391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1134/2277] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:51.088971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.089249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.089520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.108908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:51.110212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.118115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1135/2278] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:51.121981 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.131452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4ff2a-748c-43ed-9705-5894ca91ad42 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.136354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.136927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.137206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.137473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.138140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:51.138435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.140180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.155306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1136/2279] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:14:51.156296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.163881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f5ee18b63c47cfa7ce789bd004f3b9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.164593 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f5ee18b63c47cfa7ce789bd004f3b9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.166046 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca146d6-b4b7-4e72-b766-d4f66de30e9e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.171589 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.172240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.172523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.172797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.173421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.175359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.215877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f5ee18b63c47cfa7ce789bd004f3b9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.228698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f5ee18b63c47cfa7ce789bd004f3b9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.229430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-31a24fc6-d1b2-4449-b9d6-21e6ba42ccc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f5ee18b63c47cfa7ce789bd004f3b9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:51.243692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1137/2280] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:51.244430 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1144/2281] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:50 2022] DELETE /identity/v3/users/05f5ee18b63c47cfa7ce789bd004f3b9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:51.244679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.246260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.249215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.249498 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.249768 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.268937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:51.270137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.310740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8c974-5418-4a9c-a07d-26568c8cc41e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.329186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1145/2282] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:51.338779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.340202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.360157 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:51.366837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.367484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.367760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.368070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.374425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.374425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.397316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a58d07-036e-4870-b39b-494773a69083 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.400715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.401320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.401600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.401873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.410582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.411982 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.421813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1146/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:51.447332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1138/2284] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:51.449470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.459332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5842e-995c-4229-9267-d7c7193ccc2a service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.464327 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.464926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.465238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.465513 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.466750 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.468338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.478737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1147/2285] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:51.483458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.493742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f638e-59bc-488b-bdce-17538c7dde54 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.499330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f2102-6e27-42fa-816a-6caa74cc8edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.499967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f2102-6e27-42fa-816a-6caa74cc8edc admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.500282 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f2102-6e27-42fa-816a-6caa74cc8edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.500554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f2102-6e27-42fa-816a-6caa74cc8edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.501307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f2102-6e27-42fa-816a-6caa74cc8edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.502928 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f2102-6e27-42fa-816a-6caa74cc8edc admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.513192 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1139/2286] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:51.517762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.527697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bb38e-0836-4f43-90b0-675f325d6a8f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.532375 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.533019 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.533291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.533559 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] PATH_INFO: `/v3/projects/e7a0068484bc4ff3a58ce5b40bc65734/users/27d34f31423541568a0ffbd818f42557/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.547718 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a0068484bc4ff3a58ce5b40bc65734, user_id=27d34f31423541568a0ffbd818f42557, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.550414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.656627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1148/2287] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:51 2022] PUT /identity/v3/projects/e7a0068484bc4ff3a58ce5b40bc65734/users/27d34f31423541568a0ffbd818f42557/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:51.667083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.667343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.667618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.688728 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:51.697511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] MFA Rules not processed for user `27d34f31423541568a0ffbd818f42557`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:51.699281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.768726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1140/2288] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:51.769461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.833183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89bbc-f90a-4874-af69-284f4b6d3213 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.837815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.838416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.838692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.838965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.839584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.841476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.891955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:51.895071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:51.914297 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a8c6-0175-4b5e-a4f1-b4e92a6de594 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:51.918966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:51.920108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:51.920388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:51.920658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] PATH_INFO: `/v3/projects/a9ddc2babd074a589af9378974450895` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:51.921882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9ddc2babd074a589af9378974450895)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:51.927372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:51.949272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1149/2289] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:51.953220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.013922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461081-d4af-45c0-9fb8-2840bab10e74 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.026132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:52.028626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9ddc2babd074a589af9378974450895', 'request_id': 'req-648b9914-7ef3-49a3-9a53-25c323412c8d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:52.034400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9ddc2babd074a589af9378974450895', 'request_id': 'req-648b9914-7ef3-49a3-9a53-25c323412c8d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:52.040975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9ddc2babd074a589af9378974450895', 'request_id': 'req-648b9914-7ef3-49a3-9a53-25c323412c8d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:52.073184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.073792 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.074075 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.074351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.074994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.076945 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.083937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] The token cache is being invalidate because project a9ddc2babd074a589af9378974450895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:52.084260 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:52.084908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-648b9914-7ef3-49a3-9a53-25c323412c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:52.086591 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1141/2290] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:51 2022] DELETE /identity/v3/projects/a9ddc2babd074a589af9378974450895 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:52.091281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a8c1-5d3e-4488-9847-37a22d066f59 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.097872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a8c1-5d3e-4488-9847-37a22d066f59 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.097872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a8c1-5d3e-4488-9847-37a22d066f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.112617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1e84a8c1-5d3e-4488-9847-37a22d066f59 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:52.113803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1e84a8c1-5d3e-4488-9847-37a22d066f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:52.121625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1142/2291] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:52.129737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.132274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.132274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.148818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:52.149999 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:52.158250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1143/2292] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:52.162241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.167538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1150/2293] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:51 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:52.171942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc396e22-d3f1-4dae-8b14-4e69bc012e36 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.174955 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.176613 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:52.176814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.177431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.177712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.177986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.178639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:14:52.178946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.180638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.189302 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9fd0c-11cb-4dac-b24b-92750afd595a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.193521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1144/2294] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:14:52.194032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.194739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.195020 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.195288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.196216 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.197978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.198146 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.208919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1151/2295] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:52.212072 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0222f9b-a0bd-46f3-9526-c588439b2cb3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.212810 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.217307 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.217905 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.218176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.218453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.219089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.221003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.223320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31122d67-f698-4ad4-ab6e-7755039d041c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.228254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.228852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.229132 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.229403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.230032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.231586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.242330 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1152/2296] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:52.252802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.262635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e9a62-515a-4307-9f61-f385262aabd7 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.267295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6574253-592d-4320-875c-e4453c3a4ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.267911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6574253-592d-4320-875c-e4453c3a4ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.268227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6574253-592d-4320-875c-e4453c3a4ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.268806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6574253-592d-4320-875c-e4453c3a4ebc admin admin] PATH_INFO: `/v3/projects/6222043dd7ec420eb97f2c74bac66ed3/users/cd7b08bdfcd042b8aee2e8be3dda7885/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.284587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6574253-592d-4320-875c-e4453c3a4ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6222043dd7ec420eb97f2c74bac66ed3, user_id=cd7b08bdfcd042b8aee2e8be3dda7885, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.289355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1145/2297] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:14:52.289848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6574253-592d-4320-875c-e4453c3a4ebc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.290600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.325279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:52.335755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1153/2298] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:52 2022] PUT /identity/v3/projects/6222043dd7ec420eb97f2c74bac66ed3/users/cd7b08bdfcd042b8aee2e8be3dda7885/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:52.347711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.347994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.348292 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.354206 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563d484-324d-4f15-bff5-40f7e961d1b2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.376604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] MFA Rules not processed for user `cd7b08bdfcd042b8aee2e8be3dda7885`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:52.377827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:52.411335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.412657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.412657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.412657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.413160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.415195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.443329 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1154/2299] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:52.447767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.457865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded08bc-c0f1-4636-842d-221c26f879e6 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.462326 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.462904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.463180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.463452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.464187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.466081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.499701 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1146/2300] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:52.500872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.510369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf474542-23e5-435f-9dd4-5a28ef65654e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.515463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.516045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.516331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.516602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.517226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.518826 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.528856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1147/2301] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:14:52.532486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.541527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf729bf-9c7a-4bec-8d1e-37544cd907d1 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.545997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.546551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.546829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.547096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.547698 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.551797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1155/2302] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:52.555494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.555650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.563695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1148/2303] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:52.564665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.580666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb3840-5cc2-471e-b8b7-fc8486a9b0af admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.585883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:52.617274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd9674-0360-4c4e-948c-abcdc4a30f1a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.622021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.623311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.623590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.623856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.624683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.626570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.637707 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4287a2-0f81-41e6-937f-3a1530192357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.638383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4287a2-0f81-41e6-937f-3a1530192357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.638692 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4287a2-0f81-41e6-937f-3a1530192357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.638970 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4287a2-0f81-41e6-937f-3a1530192357 admin admin] PATH_INFO: `/v3/projects/4f9731af8c4c4188bc61c7f35b8bdc5e/users/e750af16ce0d4742a3305141470ff308/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.652904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4287a2-0f81-41e6-937f-3a1530192357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9731af8c4c4188bc61c7f35b8bdc5e, user_id=e750af16ce0d4742a3305141470ff308, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.655515 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4287a2-0f81-41e6-937f-3a1530192357 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.708655 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1156/2304] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:52.712541 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.718482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1149/2305] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:52 2022] PUT /identity/v3/projects/4f9731af8c4c4188bc61c7f35b8bdc5e/users/e750af16ce0d4742a3305141470ff308/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:52.723086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.723446 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.723811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.725796 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5c7e2-8c06-4f01-8f2d-ea15769b17a2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.743487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:52.744734 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:52.770845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:52.782282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.782888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.783173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.783481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.784109 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.785689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.799009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1157/2306] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:52.800790 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.804317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1150/2307] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:52.810966 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363783b-8b7f-43aa-a0c6-72b992e9abfe admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.814427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.815641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.815803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:52.816336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.816620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.816894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] PATH_INFO: `/v3/projects/1fe7c8d559644a698d91ac722f906cac/users/1e61d2def61f46a58e7859ca64f61ba8/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.831991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe7c8d559644a698d91ac722f906cac, user_id=1e61d2def61f46a58e7859ca64f61ba8, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.834687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.871658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e4253-b2e4-4cd7-b0b2-4e91bdc08c23 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.875889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.876524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.876804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.877076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.885987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.887405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.896893 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1151/2308] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:14:52.901580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1158/2309] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:52 2022] PUT /identity/v3/projects/1fe7c8d559644a698d91ac722f906cac/users/1e61d2def61f46a58e7859ca64f61ba8/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:52.905470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:52.914877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f4ccb-966d-463b-8199-c13765b8113f service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:52.971838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730852b-0073-495d-8655-e69e7d8f9f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:52.972410 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730852b-0073-495d-8655-e69e7d8f9f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:52.972675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730852b-0073-495d-8655-e69e7d8f9f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:52.972942 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730852b-0073-495d-8655-e69e7d8f9f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:52.973614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730852b-0073-495d-8655-e69e7d8f9f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:52.975179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730852b-0073-495d-8655-e69e7d8f9f60 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:52.985218 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1152/2310] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:52.989733 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:53.000873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afdcd2-f637-466e-9945-200c04d44ec9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:53.008234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:53.008938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:53.009219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:53.009495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] PATH_INFO: `/v3/projects/1fe7c8d559644a698d91ac722f906cac/users/1e61d2def61f46a58e7859ca64f61ba8/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:53.011562 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe7c8d559644a698d91ac722f906cac, user_id=1e61d2def61f46a58e7859ca64f61ba8, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:53.014875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:53.025578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1159/2311] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:52 2022] PUT /identity/v3/projects/1fe7c8d559644a698d91ac722f906cac/users/1e61d2def61f46a58e7859ca64f61ba8/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:53.036111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174c953-0083-4733-bcfa-48899f466243 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:53.036514 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174c953-0083-4733-bcfa-48899f466243 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:53.036791 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174c953-0083-4733-bcfa-48899f466243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:53.067686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-e174c953-0083-4733-bcfa-48899f466243 None None] MFA Rules not processed for user `1e61d2def61f46a58e7859ca64f61ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:53.068945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e174c953-0083-4733-bcfa-48899f466243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:53.129844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:53.138838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1153/2312] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:53.150146 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:53.151830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:53.218106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea9b8e-3ffb-4915-be5e-834379ea3ee2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:53.223023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:53.223023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:53.223023 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:53.223335 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:53.234397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:53.235833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:53.246263 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1160/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:53.374764 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174c953-0083-4733-bcfa-48899f466243 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:53.376271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e174c953-0083-4733-bcfa-48899f466243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:53.388353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174c953-0083-4733-bcfa-48899f466243 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:53.392201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:53.392816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:53.393094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:53.394407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:53.395289 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:53.461650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:53.461650 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879611a-d822-4239-8785-c999f33ff63d service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:53.469790 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1154/2314] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:57.555207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:57.581199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:57.625904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993d658-f84d-4090-9461-48603fb4adc4 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:57.630621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:57.631265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:57.631548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:57.631825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] PATH_INFO: `/v3/users/27d34f31423541568a0ffbd818f42557` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:57.633003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d34f31423541568a0ffbd818f42557)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:57.635299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:57.879815 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d34f31423541568a0ffbd818f42557'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:57.880660 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d34f31423541568a0ffbd818f42557'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:57.954382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d34f31423541568a0ffbd818f42557'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:57.971813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d34f31423541568a0ffbd818f42557'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:57.972811 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d34f31423541568a0ffbd818f42557'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:57.995260 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1161/2315] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:57 2022] DELETE /identity/v3/users/27d34f31423541568a0ffbd818f42557 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:14:58.000628 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a378f-4c76-45fe-8619-c6e97ad08015 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.001138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a378f-4c76-45fe-8619-c6e97ad08015 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.001450 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a378f-4c76-45fe-8619-c6e97ad08015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.027811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-0a8a378f-4c76-45fe-8619-c6e97ad08015 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:58.029242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-0a8a378f-4c76-45fe-8619-c6e97ad08015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:58.104179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1155/2316] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:14:58.124824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.124824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:58.126421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.126646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.126925 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.172965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] MFA Rules not processed for user `e750af16ce0d4742a3305141470ff308`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:14:58.175076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:58.199589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fc7fb-018d-499b-b7fd-e59a716bfd03 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.203871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.204537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.204817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.205087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.215433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.216976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.227685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1162/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:14:58.255779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1156/2318] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:14:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:14:58.260218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.261674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:58.275867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c744d5f-37aa-4d09-a008-1e9df270f795 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.281249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.281909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.282314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.282848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.283633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.285795 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.367758 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1163/2319] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:14:58 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:58.371376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.383505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893d286-d31f-469e-8b19-ba611ed93338 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.405426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:58.444015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.444615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.444896 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.445166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.446076 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.448033 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.555503 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1157/2320] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:14:58 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:14:58.559313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.570950 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5e20e-5b1a-40f2-a918-bc24e3067421 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.576368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.577671 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.577959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.578533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.579107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.580909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.594249 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1164/2321] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:58.598141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.610089 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c480df-e1f1-44d0-8264-2f48b1173000 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.614803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.615468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.615751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.616041 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] PATH_INFO: `/v3/projects/dfaa60d75ad34d5dbe9402bb99667133/users/6d1303f35bb24d56880116afa911467e/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.632013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfaa60d75ad34d5dbe9402bb99667133, user_id=6d1303f35bb24d56880116afa911467e, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.635516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.680303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.682104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:58.695401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e78c1-c7ea-42ce-b888-7c25fd6eca8a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.700417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.701638 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.701920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.702218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] PATH_INFO: `/v3/projects/e7a0068484bc4ff3a58ce5b40bc65734` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.703367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a0068484bc4ff3a58ce5b40bc65734)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.705315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.711934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:58.738776 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1158/2322] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:58 2022] PUT /identity/v3/projects/dfaa60d75ad34d5dbe9402bb99667133/users/6d1303f35bb24d56880116afa911467e/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:58.742669 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.806442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aabdb1-f31a-4881-80da-ab5897545169 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.811139 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.811735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.812016 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.812318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.813066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.814794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.826778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1159/2323] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:14:58.832965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.836772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a0068484bc4ff3a58ce5b40bc65734', 'request_id': 'req-18252160-3d33-48d2-9ef1-6693a601a481'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:58.845494 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369775f3-c83d-461a-aece-14aeac839e60 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:58.907091 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:58.907752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:58.908050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:58.908340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] PATH_INFO: `/v3/projects/dfaa60d75ad34d5dbe9402bb99667133/users/6d1303f35bb24d56880116afa911467e/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:58.910253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfaa60d75ad34d5dbe9402bb99667133, user_id=6d1303f35bb24d56880116afa911467e, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:58.913058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:58.918981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a0068484bc4ff3a58ce5b40bc65734', 'request_id': 'req-18252160-3d33-48d2-9ef1-6693a601a481'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:58.925382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1160/2324] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:14:58 2022] PUT /identity/v3/projects/dfaa60d75ad34d5dbe9402bb99667133/users/6d1303f35bb24d56880116afa911467e/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:58.935845 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a0068484bc4ff3a58ce5b40bc65734', 'request_id': 'req-18252160-3d33-48d2-9ef1-6693a601a481'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:58.945779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] The token cache is being invalidate because project e7a0068484bc4ff3a58ce5b40bc65734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:14:58.946317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:58.947021 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:58.948614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1165/2325] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:14:58 2022] DELETE /identity/v3/projects/e7a0068484bc4ff3a58ce5b40bc65734 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:14:58.953316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:14:58.955088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:14:58.983604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:14:59.030129 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5000e3-8313-47c5-8eb1-80affbf19430 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:14:59.034911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:14:59.035540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:14:59.035814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:14:59.036108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] PATH_INFO: `/v3/users/cd7b08bdfcd042b8aee2e8be3dda7885` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:14:59.037251 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7b08bdfcd042b8aee2e8be3dda7885)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:14:59.039273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:14:59.238386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7b08bdfcd042b8aee2e8be3dda7885'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:59.284956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7b08bdfcd042b8aee2e8be3dda7885'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:59.285570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7b08bdfcd042b8aee2e8be3dda7885'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:59.286536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7b08bdfcd042b8aee2e8be3dda7885'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:59.302860 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7b08bdfcd042b8aee2e8be3dda7885'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:14:59.320127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1161/2326] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:14:58 2022] DELETE /identity/v3/users/cd7b08bdfcd042b8aee2e8be3dda7885 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:00.030649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:00.041814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18252160-3d33-48d2-9ef1-6693a601a481 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:00.127584 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:00.128492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:00.128875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:00.129256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] PATH_INFO: `/v3/projects/6222043dd7ec420eb97f2c74bac66ed3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:00.130505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6222043dd7ec420eb97f2c74bac66ed3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:00.132620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:00.252150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6222043dd7ec420eb97f2c74bac66ed3', 'request_id': 'req-b0221f42-e072-4253-957b-0721a38ea924'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.370695 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6222043dd7ec420eb97f2c74bac66ed3', 'request_id': 'req-b0221f42-e072-4253-957b-0721a38ea924'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.410912 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6222043dd7ec420eb97f2c74bac66ed3', 'request_id': 'req-b0221f42-e072-4253-957b-0721a38ea924'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.437976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] The token cache is being invalidate because project 6222043dd7ec420eb97f2c74bac66ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:15:00.438266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.440935 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.449370 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1166/2327] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:15:00 2022] DELETE /identity/v3/projects/6222043dd7ec420eb97f2c74bac66ed3 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:00.456321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:00.457846 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:00.473233 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f20e5-8b10-4c61-b97f-80cb7ec192a3 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:00.549611 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:00.550250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:00.550573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:00.550851 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] PATH_INFO: `/v3/users/1e61d2def61f46a58e7859ca64f61ba8` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:00.552519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e61d2def61f46a58e7859ca64f61ba8)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:00.554693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:00.782880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:00.793155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e61d2def61f46a58e7859ca64f61ba8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.896575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e61d2def61f46a58e7859ca64f61ba8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.897258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e61d2def61f46a58e7859ca64f61ba8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.898126 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e61d2def61f46a58e7859ca64f61ba8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.924377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-36a2d798-f04f-45a6-9d40-2cea412275ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e61d2def61f46a58e7859ca64f61ba8'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:00.973536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1162/2328] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:15:00 2022] DELETE /identity/v3/users/1e61d2def61f46a58e7859ca64f61ba8 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:01.684133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:01.685878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:01.706328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0221f42-e072-4253-957b-0721a38ea924 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:01.740595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:01.800185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:01.801112 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:01.801452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:01.801736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] PATH_INFO: `/v3/projects/1fe7c8d559644a698d91ac722f906cac` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:01.803298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe7c8d559644a698d91ac722f906cac)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:01.805743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:01.957700 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe7c8d559644a698d91ac722f906cac', 'request_id': 'req-d4b561e2-0535-4570-bd86-afc3ace42c14'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:02.056224 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe7c8d559644a698d91ac722f906cac', 'request_id': 'req-d4b561e2-0535-4570-bd86-afc3ace42c14'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:02.064118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe7c8d559644a698d91ac722f906cac', 'request_id': 'req-d4b561e2-0535-4570-bd86-afc3ace42c14'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:02.081320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] The token cache is being invalidate because project 1fe7c8d559644a698d91ac722f906cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:15:02.081626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:02.082439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d4b561e2-0535-4570-bd86-afc3ace42c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:02.084274 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1167/2329] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:15:01 2022] DELETE /identity/v3/projects/1fe7c8d559644a698d91ac722f906cac => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:02.090050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.090560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.090665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.118330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:02.124745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:02.198675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1163/2330] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:02.207983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.208368 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.208985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.235620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:02.237027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:02.248180 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1168/2331] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:15:02.252217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:02.263071 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680fbe5-7872-4877-8a34-7418942aeabe None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:02.270428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:02.271109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.271549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.282470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.282470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:15:02.282470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:02.282470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:02.294162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1164/2332] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:15:02.295030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:02.305692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda65876-68f2-4c23-8f4e-043493f775a9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:02.314004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:02.314643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.314919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.315189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.315837 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:02.322938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:02.426600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1169/2333] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:15:02 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:15:02.430814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:02.442058 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092690fc-4bdf-482a-920a-a25e24fae294 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:02.521760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:02.522449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.522737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.523011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.523704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:02.532537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:02.614403 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1165/2334] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:15:02 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:15:02.618254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:02.636201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38b10e-c2cc-40ba-985c-b6e834a3619a admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:02.648204 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:02.648969 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.649331 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.649736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.650466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:02.652822 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:02.674771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1170/2335] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:02.678974 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:02.693745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505948a-4c44-43c3-b854-849c4c9ba898 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:02.699600 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3760ea-fd49-4953-91b7-112d413f67bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:02.700319 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3760ea-fd49-4953-91b7-112d413f67bf admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.700594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3760ea-fd49-4953-91b7-112d413f67bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.700876 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3760ea-fd49-4953-91b7-112d413f67bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.701534 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3760ea-fd49-4953-91b7-112d413f67bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:02.703316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3760ea-fd49-4953-91b7-112d413f67bf admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:02.717495 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1166/2336] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:02.722940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:02.740464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abada80f-7d3e-4e17-97c5-25152b0af2c1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:02.753380 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:02.754867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.755376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.755865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] PATH_INFO: `/v3/projects/30a4c35ea9824d6c99d7746ae80f769b/users/8682c46f10634e3cb37c4373108e140d/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.777918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a4c35ea9824d6c99d7746ae80f769b, user_id=8682c46f10634e3cb37c4373108e140d, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:02.782726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:02.857850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1171/2337] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:15:02 2022] PUT /identity/v3/projects/30a4c35ea9824d6c99d7746ae80f769b/users/8682c46f10634e3cb37c4373108e140d/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:02.930951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534994f-557e-4662-b76d-5b441b93331f None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:02.930951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534994f-557e-4662-b76d-5b441b93331f None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:02.930951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534994f-557e-4662-b76d-5b441b93331f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:02.964536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9534994f-557e-4662-b76d-5b441b93331f None None] MFA Rules not processed for user `8682c46f10634e3cb37c4373108e140d`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:02.965865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9534994f-557e-4662-b76d-5b441b93331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:02.980510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:03.053932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1167/2338] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:03.055188 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:03.144045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:03.170202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ddcfe-cbdd-451c-9579-14e0234ed6c2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:03.178569 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:03.179314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:03.179602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:03.180100 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:03.180807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:03.183045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:03.361053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1172/2339] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:15:03 2022] POST /identity/v3/projects => generated 385 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:15:03.365240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534994f-557e-4662-b76d-5b441b93331f None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:03.378915 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534994f-557e-4662-b76d-5b441b93331f None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:03.454431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:03.455402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:03.455998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:03.456498 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:03.457441 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:03.459845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:03.579420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1168/2340] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:15:03 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:15:03.581182 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:03.597958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67764fa-1dfa-40fe-a96d-985d3c604943 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:03.605685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:03.607123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:03.607123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:03.609115 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:03.610378 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:03.613343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:03.636853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1173/2341] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:03.642775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:03.662228 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399546a-8bc2-4122-9f6a-b3135d61f282 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:03.671338 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00f6cc-2d5b-461f-bbcc-bad2a71dbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:03.672722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f6cc-2d5b-461f-bbcc-bad2a71dbc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:03.673255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f6cc-2d5b-461f-bbcc-bad2a71dbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:03.674044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f6cc-2d5b-461f-bbcc-bad2a71dbc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:03.676582 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00f6cc-2d5b-461f-bbcc-bad2a71dbc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:03.678960 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00f6cc-2d5b-461f-bbcc-bad2a71dbc99 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:03.699349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1169/2342] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:03.703895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:03.720645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dbf4a-48f4-4567-82f3-b1c6fb083b47 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:03.735414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:03.736253 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:03.736417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:03.736694 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] PATH_INFO: `/v3/projects/d1809269b71648f9bd18c77022824641/users/ac4bfa33a1004b9cbeeb5609d282dde5/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:03.776087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1809269b71648f9bd18c77022824641, user_id=ac4bfa33a1004b9cbeeb5609d282dde5, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:03.778854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:03.885185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1174/2343] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:15:03 2022] PUT /identity/v3/projects/d1809269b71648f9bd18c77022824641/users/ac4bfa33a1004b9cbeeb5609d282dde5/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:03.896675 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:03.897606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:03.898142 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:03.964691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] MFA Rules not processed for user `ac4bfa33a1004b9cbeeb5609d282dde5`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:03.966331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:04.082954 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1170/2344] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:04.087050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:04.159921 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a5534-c26e-46f6-afbc-a192cd25484b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:04.165964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:04.166598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.166874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.167715 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.168125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:04.172213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:04.291824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1175/2345] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:15:04 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:15:04.292322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:04.308607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff0c6f-8ab8-4f6e-b07c-370ee596b703 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:04.363114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:04.409351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:04.410250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.410541 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.411350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.411489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:04.413599 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:04.542454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1171/2346] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:15:04 2022] POST /identity/v3/users => generated 299 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:15:04.543752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:04.556476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26973eea-77ba-4d64-a2d5-47cb44cc94a2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:04.573791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:04.574864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.575220 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.575620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.576916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:04.590625 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:04.606017 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1176/2347] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:04.607741 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:04.619496 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fcd3-d861-44d7-9d3e-8850452313d5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:04.625688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:04.626351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.626694 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.626972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] PATH_INFO: `/v3/projects/4f04ca2bb2fb4e3ba8f89755e009f4ca/users/faf0a7cd0be54d3eacdca866377e7d32/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.643343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f04ca2bb2fb4e3ba8f89755e009f4ca, user_id=faf0a7cd0be54d3eacdca866377e7d32, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:04.646465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:04.716065 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1172/2348] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:15:04 2022] PUT /identity/v3/projects/4f04ca2bb2fb4e3ba8f89755e009f4ca/users/faf0a7cd0be54d3eacdca866377e7d32/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:04.718756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:04.729516 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ae6f7-6ebd-4b8b-a7b8-29cbf0bb054f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:04.827258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501088f6-9a4b-4042-8014-ba79b4006670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:04.828937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501088f6-9a4b-4042-8014-ba79b4006670 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.829643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501088f6-9a4b-4042-8014-ba79b4006670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.830080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501088f6-9a4b-4042-8014-ba79b4006670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.831409 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501088f6-9a4b-4042-8014-ba79b4006670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:04.834126 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501088f6-9a4b-4042-8014-ba79b4006670 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:04.853631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1177/2349] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:04.857737 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:04.875704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d869-3fd4-451d-9a91-f1ed929fa7cd admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:04.890062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:04.891003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.891288 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.891799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] PATH_INFO: `/v3/projects/4f04ca2bb2fb4e3ba8f89755e009f4ca/users/faf0a7cd0be54d3eacdca866377e7d32/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.895294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f04ca2bb2fb4e3ba8f89755e009f4ca, user_id=faf0a7cd0be54d3eacdca866377e7d32, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:04.902718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:04.931052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1173/2350] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:15:04 2022] PUT /identity/v3/projects/4f04ca2bb2fb4e3ba8f89755e009f4ca/users/faf0a7cd0be54d3eacdca866377e7d32/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:04.943431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1f4d6-f58c-4d31-9154-a6fb5949cb8d None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:04.948604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1f4d6-f58c-4d31-9154-a6fb5949cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:04.948604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1f4d6-f58c-4d31-9154-a6fb5949cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:04.979607 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:04.991528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d6f1f4d6-f58c-4d31-9154-a6fb5949cb8d None None] MFA Rules not processed for user `faf0a7cd0be54d3eacdca866377e7d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:04.993150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d6f1f4d6-f58c-4d31-9154-a6fb5949cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:05.101874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1178/2351] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:05.114414 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:05.116792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:05.199124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67209764-4d17-4b5e-9ac6-255b9227ae41 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:05.203517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:05.204517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:05.205045 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:05.205558 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:05.282426 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:05.285061 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:05.295032 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1174/2352] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:06.051528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62761e-2cca-4e4f-8185-34eb7ae0f65e None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:06.051989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62761e-2cca-4e4f-8185-34eb7ae0f65e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:06.052169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62761e-2cca-4e4f-8185-34eb7ae0f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:06.093387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-8a62761e-2cca-4e4f-8185-34eb7ae0f65e None None] MFA Rules not processed for user `6d1303f35bb24d56880116afa911467e`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:06.094618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-8a62761e-2cca-4e4f-8185-34eb7ae0f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:06.181573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1179/2353] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:06.191458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:06.193486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:06.229458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:06.257366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff5238-9817-40a3-b3c8-3b6d9f3953fa service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:06.261247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe6839ce3ac4e6eb27b54eb1870c67d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:06.261959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:06.262285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:06.262658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:06.264811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:06.331856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:06.333342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:06.342145 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1175/2354] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:06.611919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794052b0-d244-4633-b02d-08c426557afe None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:06.612235 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794052b0-d244-4633-b02d-08c426557afe None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:06.616563 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794052b0-d244-4633-b02d-08c426557afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:06.642605 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-794052b0-d244-4633-b02d-08c426557afe None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:06.643801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-794052b0-d244-4633-b02d-08c426557afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:06.711318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1180/2355] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:15:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:06.739038 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:06.740527 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:06.821797 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601a81d-c9e6-4c9e-a44a-b16f7c954ef5 service glance] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:06.827244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:06.827907 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:06.828215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:06.828505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:06.838861 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:06.840512 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:06.852645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1176/2356] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:15:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:15:06.872120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794052b0-d244-4633-b02d-08c426557afe None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:06.872120 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-794052b0-d244-4633-b02d-08c426557afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:06.897156 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:06.940305 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794052b0-d244-4633-b02d-08c426557afe None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:06.943937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:06.944648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:06.944934 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:06.945213 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:06.947687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:07.023208 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:07.024695 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:07.036799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1181/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:07.542711 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:07.560882 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48090-2a49-4ca5-9540-234829f3497e service swift] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:07.573690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:07.574662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:07.575092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:07.576185 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:07.693413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:07.696470 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:07.710871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1177/2358] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:09.621648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:09.638839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27211e-6831-4e15-a2d1-bf69b00f0854 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:09.651339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:09.652207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:09.652641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:09.653064 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:09.664652 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:09.666461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:09.688173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1182/2359] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:12.789253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:12.790710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:12.815817 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f829-b4b8-4e7f-ad0b-0f090b02f5ea service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:12.826433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:12.827321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:12.830240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:12.830723 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] PATH_INFO: `/v3/users/b8507de448b6455fb792bda0c949421d` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:12.833842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8507de448b6455fb792bda0c949421d)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:12.841217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:12.852479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:13.067506 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8507de448b6455fb792bda0c949421d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:13.165720 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8507de448b6455fb792bda0c949421d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:13.166637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8507de448b6455fb792bda0c949421d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:13.167271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8507de448b6455fb792bda0c949421d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:13.183849 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-ea45dc85-c0a0-4f87-9b20-133e7a76688f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8507de448b6455fb792bda0c949421d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:13.204079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1178/2360] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:15:12 2022] DELETE /identity/v3/users/b8507de448b6455fb792bda0c949421d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:13.588314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:13.611909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe13a2-90f5-403f-abb0-7a6094088173 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:13.743382 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:13.744407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:13.744850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:13.745353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] PATH_INFO: `/v3/projects/34f584f725354f68b917b45b645b3630` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:13.746726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f584f725354f68b917b45b645b3630)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:13.749354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:13.894084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f584f725354f68b917b45b645b3630', 'request_id': 'req-0b05e1ae-228e-4316-8943-58728d9cf7ab'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:13.895140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:13.999320 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f584f725354f68b917b45b645b3630', 'request_id': 'req-0b05e1ae-228e-4316-8943-58728d9cf7ab'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:14.007897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f584f725354f68b917b45b645b3630', 'request_id': 'req-0b05e1ae-228e-4316-8943-58728d9cf7ab'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:14.023772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] The token cache is being invalidate because project 34f584f725354f68b917b45b645b3630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:15:14.024242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:14.025014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0b05e1ae-228e-4316-8943-58728d9cf7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:14.026925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1183/2361] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:15:13 2022] DELETE /identity/v3/projects/34f584f725354f68b917b45b645b3630 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:14.031802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.032346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.032451 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.056796 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:14.058396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:14.137638 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1179/2362] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:14.140403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.141053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.141543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.166325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:14.167926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:14.179489 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1184/2363] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:14.184083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:14.195111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7212521-1b8d-4af5-b5a3-d0ca71c015a3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:14.200284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:14.201643 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.202093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.202547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.203376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:15:14.203682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:14.205305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:14.221521 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1180/2364] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:15:14.226036 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:14.237779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816371e-f929-4a6e-9b41-c8aa55ea20de None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:14.242524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:14.243177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.243455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.243722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.244381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:14.246355 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:14.322601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1185/2365] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:15:14 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:15:14.323395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:14.336994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2af23-b0e3-4830-8833-535529ed677f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:14.410897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:14.411398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.411682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.411953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.412624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:14.414651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:14.486249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:14.496662 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1181/2366] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:15:14 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:15:14.498372 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:14.511780 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a873-cd86-482f-8c9d-e88f60e608c1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:14.521189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:14.522060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.522402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.522830 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.523597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:14.525635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:14.551340 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1186/2367] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:15:14.555753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:14.574792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d6da2-09f4-4e85-af05-9c8da919716f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:14.584096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:14.584096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.584096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.584096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.584664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:14.584664 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:14.595771 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1182/2368] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:14.600826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:14.612473 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10a91e-8a38-486f-a301-e59efb7a03c9 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:14.619651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68c61-0c41-446b-8d66-df3d907d2952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:14.620633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68c61-0c41-446b-8d66-df3d907d2952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:14.620997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68c61-0c41-446b-8d66-df3d907d2952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:14.621348 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68c61-0c41-446b-8d66-df3d907d2952 admin admin] PATH_INFO: `/v3/projects/f4fa5392cfd049509a86b2d88d536409/users/50d98bc599f74688b7f9fea92ee7c2ea/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:14.646105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68c61-0c41-446b-8d66-df3d907d2952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fa5392cfd049509a86b2d88d536409, user_id=50d98bc599f74688b7f9fea92ee7c2ea, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:14.649744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68c61-0c41-446b-8d66-df3d907d2952 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:15.056929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:15.058758 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:15.072395 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a3d6-ffde-4867-a177-c7a5521420b7 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:15.076901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:15.077524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:15.077799 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:15.078070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] PATH_INFO: `/v3/users/8682c46f10634e3cb37c4373108e140d` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:15.079174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8682c46f10634e3cb37c4373108e140d)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:15.081119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:15.142475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1187/2369] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:15:14 2022] PUT /identity/v3/projects/f4fa5392cfd049509a86b2d88d536409/users/50d98bc599f74688b7f9fea92ee7c2ea/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:15.147259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:15.147540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:15.147807 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:15.166351 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:15.167538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:15.225374 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1188/2370] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:15:15.235300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:15.238058 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:15.299451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a3428-3ad9-471e-9227-d873e8e71e49 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:15.303650 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d627-2e06-4f2f-817b-05bea11f9ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:15.304487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d627-2e06-4f2f-817b-05bea11f9ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:15.304752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d627-2e06-4f2f-817b-05bea11f9ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:15.305027 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d627-2e06-4f2f-817b-05bea11f9ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:15.316145 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d627-2e06-4f2f-817b-05bea11f9ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:15.317765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d627-2e06-4f2f-817b-05bea11f9ea4 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:15.329129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1189/2371] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:15:15.337274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8682c46f10634e3cb37c4373108e140d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:15.387274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8682c46f10634e3cb37c4373108e140d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:15.388476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8682c46f10634e3cb37c4373108e140d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:15.389137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8682c46f10634e3cb37c4373108e140d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:15.403394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8682c46f10634e3cb37c4373108e140d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:15.420101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1183/2372] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:15:15 2022] DELETE /identity/v3/users/8682c46f10634e3cb37c4373108e140d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:15:15.424315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:15.424591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:15.424865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:15.450271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:15.452052 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:15.475894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:15.549227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1190/2373] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:15.559028 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:15.561508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:15.689088 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1061cc2-3c24-48eb-bd16-1aa884764a4e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:15.694749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:15.695424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:15.695713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:15.696293 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:15.709786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:15.711240 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:15.731012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1184/2374] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:15:16.261275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:16.263870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:16.285215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1b78-8288-4694-8fa9-e15b252b8f56 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:16.290448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:16.291359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:16.291842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:16.292247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] PATH_INFO: `/v3/projects/30a4c35ea9824d6c99d7746ae80f769b` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:16.293478 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a4c35ea9824d6c99d7746ae80f769b)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:16.297127 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:16.433095 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a4c35ea9824d6c99d7746ae80f769b', 'request_id': 'req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.523502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a4c35ea9824d6c99d7746ae80f769b', 'request_id': 'req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.523502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a4c35ea9824d6c99d7746ae80f769b', 'request_id': 'req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.541070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] The token cache is being invalidate because project 30a4c35ea9824d6c99d7746ae80f769b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:15:16.541500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.542394 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.543968 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1191/2375] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:15:16 2022] DELETE /identity/v3/projects/30a4c35ea9824d6c99d7746ae80f769b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:16.549594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:16.560753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a5e17-f4ba-48ca-91b7-0cfa82a86d71 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:16.637459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:16.638511 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:16.638795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:16.639067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] PATH_INFO: `/v3/users/ac4bfa33a1004b9cbeeb5609d282dde5` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:16.640525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4bfa33a1004b9cbeeb5609d282dde5)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:16.642432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:16.647945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:16.836220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4bfa33a1004b9cbeeb5609d282dde5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.908468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4bfa33a1004b9cbeeb5609d282dde5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.909183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4bfa33a1004b9cbeeb5609d282dde5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.910559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4bfa33a1004b9cbeeb5609d282dde5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.933431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4bfa33a1004b9cbeeb5609d282dde5'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:16.961349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1185/2376] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:15:16 2022] DELETE /identity/v3/users/ac4bfa33a1004b9cbeeb5609d282dde5 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:17.483250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:17.485603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:17.501553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2ab9d-6798-4dc0-bdfa-0c4b08aa7745 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:17.574273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:17.576129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:17.576129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:17.576706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] PATH_INFO: `/v3/projects/d1809269b71648f9bd18c77022824641` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:17.578074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1809269b71648f9bd18c77022824641)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:17.580285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:17.718241 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:17.719680 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1809269b71648f9bd18c77022824641', 'request_id': 'req-a9f5bc63-d854-4127-b5f1-83c249482cfc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:17.798651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1809269b71648f9bd18c77022824641', 'request_id': 'req-a9f5bc63-d854-4127-b5f1-83c249482cfc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:17.805948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1809269b71648f9bd18c77022824641', 'request_id': 'req-a9f5bc63-d854-4127-b5f1-83c249482cfc'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:17.831367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] The token cache is being invalidate because project d1809269b71648f9bd18c77022824641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:15:17.831689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:17.832686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:17.835189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1192/2377] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:15:17 2022] DELETE /identity/v3/projects/d1809269b71648f9bd18c77022824641 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:17.839476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:17.841499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:17.859768 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e23e95-77ff-4e3d-81ae-19f265da404e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:17.941938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:17.942579 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:17.942859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:17.943270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] PATH_INFO: `/v3/users/faf0a7cd0be54d3eacdca866377e7d32` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:17.944540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf0a7cd0be54d3eacdca866377e7d32)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:17.947075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:18.146542 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf0a7cd0be54d3eacdca866377e7d32'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:18.234452 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf0a7cd0be54d3eacdca866377e7d32'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:18.235148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf0a7cd0be54d3eacdca866377e7d32'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:18.235809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf0a7cd0be54d3eacdca866377e7d32'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:18.252331 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-9d977f18-84f1-451b-ac67-2c27b9cf82f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf0a7cd0be54d3eacdca866377e7d32'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:18.275663 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1186/2378] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:15:17 2022] DELETE /identity/v3/users/faf0a7cd0be54d3eacdca866377e7d32 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:19.211476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:19.212978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:19.257871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5bc63-d854-4127-b5f1-83c249482cfc admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:19.343491 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:19.351013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:19.351013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:19.351013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] PATH_INFO: `/v3/projects/4f04ca2bb2fb4e3ba8f89755e009f4ca` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:19.351013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f04ca2bb2fb4e3ba8f89755e009f4ca)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:19.351013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:19.477160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f04ca2bb2fb4e3ba8f89755e009f4ca', 'request_id': 'req-75f90523-f7e6-4a54-9574-87c3efe9d3ef'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:19.599079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f04ca2bb2fb4e3ba8f89755e009f4ca', 'request_id': 'req-75f90523-f7e6-4a54-9574-87c3efe9d3ef'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:19.608071 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f04ca2bb2fb4e3ba8f89755e009f4ca', 'request_id': 'req-75f90523-f7e6-4a54-9574-87c3efe9d3ef'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:19.624698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] The token cache is being invalidate because project 4f04ca2bb2fb4e3ba8f89755e009f4ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:15:19.624979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:19.625667 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-75f90523-f7e6-4a54-9574-87c3efe9d3ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:15:19.627910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1193/2379] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:15:19 2022] DELETE /identity/v3/projects/4f04ca2bb2fb4e3ba8f89755e009f4ca => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:19.632388 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:19.632883 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:19.632997 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:19.640751 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:19.656247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:19.659972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:19.738449 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1187/2380] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:19.740128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:19.740438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:19.740703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:19.763548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:19.764819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:19.775809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1194/2381] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:15:19.779902 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:19.791074 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6ed60-59e6-44c9-9ad6-35b692a1cf35 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:19.797209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:19.798191 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:19.798721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:19.799138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:19.799991 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:15:19.800877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:19.802318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:19.820305 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1188/2382] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:15:19.825443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:19.839875 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a083db-9d0e-48f8-994f-88648a381348 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:19.847013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:19.847922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:19.848352 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:19.849066 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:19.849801 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:19.851906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:19.988546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1195/2383] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:15:19 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:15:19.992425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:20.005238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ff4ec-2ca5-4283-9b2c-7fd1c5196eb7 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:20.110877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:20.111594 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.111873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.112548 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.112806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:20.114699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:20.223477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1189/2384] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:15:19 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:15:20.227487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:20.238118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529a16-ac3a-4d9f-b365-df418b4e0d9b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:20.243297 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:20.243848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.244583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.244583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.245084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:20.246697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:20.258779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1196/2385] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:15:20.263609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:20.274983 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc31bc8-e352-4baa-8ee9-24bf846839c4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:20.280458 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f01ca5-c90f-4679-8cdd-508df85e2bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:20.283264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f01ca5-c90f-4679-8cdd-508df85e2bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.283264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f01ca5-c90f-4679-8cdd-508df85e2bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.283264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f01ca5-c90f-4679-8cdd-508df85e2bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.283264 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f01ca5-c90f-4679-8cdd-508df85e2bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:20.284180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f01ca5-c90f-4679-8cdd-508df85e2bd9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:20.296944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1190/2386] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:15:20.301160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:20.313458 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb12fa-04d8-4584-9440-d55c9b554980 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:20.318869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:20.319407 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.319767 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.320142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] PATH_INFO: `/v3/projects/b9422ce68e164e7d8ae8dbf168e58e78/users/a800d4a53c234d2fa53eeb0c68163f44/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.349890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9422ce68e164e7d8ae8dbf168e58e78, user_id=a800d4a53c234d2fa53eeb0c68163f44, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:20.352590 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:20.454524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1197/2387] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:15:20 2022] PUT /identity/v3/projects/b9422ce68e164e7d8ae8dbf168e58e78/users/a800d4a53c234d2fa53eeb0c68163f44/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:15:20.459930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8724-a943-4a4a-8235-849fa11bbf3b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.460224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8724-a943-4a4a-8235-849fa11bbf3b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.460507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8724-a943-4a4a-8235-849fa11bbf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.492578 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-739e8724-a943-4a4a-8235-849fa11bbf3b None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:20.494303 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-739e8724-a943-4a4a-8235-849fa11bbf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:20.583943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1191/2388] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:15:20.593763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:20.595576 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:20.601722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:20.668494 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c09c37-3077-4c99-889b-95b7c5521c5b admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:20.673186 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f83872-bd13-40ba-83c4-723f0ca96070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:20.673826 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83872-bd13-40ba-83c4-723f0ca96070 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.674332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83872-bd13-40ba-83c4-723f0ca96070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.674514 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83872-bd13-40ba-83c4-723f0ca96070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.683976 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83872-bd13-40ba-83c4-723f0ca96070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:20.686181 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83872-bd13-40ba-83c4-723f0ca96070 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:20.706888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:20.707427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:20.707427 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:20.715191 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1198/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:20.727604 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] MFA Rules not processed for user `e750af16ce0d4742a3305141470ff308`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:20.728869 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:20.733729 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:20.786404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1192/2390] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:15:22.033421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27034-3c10-4712-8712-28c92c9fdb15 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:22.034776 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27034-3c10-4712-8712-28c92c9fdb15 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:22.035566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27034-3c10-4712-8712-28c92c9fdb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:22.067525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-dff27034-3c10-4712-8712-28c92c9fdb15 None None] MFA Rules not processed for user `50d98bc599f74688b7f9fea92ee7c2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:22.068814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dff27034-3c10-4712-8712-28c92c9fdb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:22.143998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1199/2391] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:15:22.160540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:22.163407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:22.253850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2f77d-bdbc-4892-a98b-96100c9df71b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:22.260148 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:22.260772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:22.261050 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:22.261324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:22.271111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:22.272897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:22.283096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1193/2392] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:22.447307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c877e-63fb-4032-bc05-5ece2092725f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:22.447531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c877e-63fb-4032-bc05-5ece2092725f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:22.447873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c877e-63fb-4032-bc05-5ece2092725f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:22.485676 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-f80c877e-63fb-4032-bc05-5ece2092725f None None] MFA Rules not processed for user `a800d4a53c234d2fa53eeb0c68163f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:22.489200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f80c877e-63fb-4032-bc05-5ece2092725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:22.596325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1200/2393] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:22.605154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:22.607193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:22.730972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194ece5-ad53-4cf9-bbb2-dffab93baade service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:22.736103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8a186-ca03-4644-b0e7-468b3ea7d780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:22.737323 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8a186-ca03-4644-b0e7-468b3ea7d780 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:22.737658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8a186-ca03-4644-b0e7-468b3ea7d780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:22.737948 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8a186-ca03-4644-b0e7-468b3ea7d780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:22.748684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8a186-ca03-4644-b0e7-468b3ea7d780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:22.750220 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8a186-ca03-4644-b0e7-468b3ea7d780 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:22.761163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1194/2394] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:24.562053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ce5e2-1c54-483d-a222-48ef32c5c776 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:24.562598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ce5e2-1c54-483d-a222-48ef32c5c776 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:24.562598 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ce5e2-1c54-483d-a222-48ef32c5c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:24.583492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-162ce5e2-1c54-483d-a222-48ef32c5c776 None None] MFA Rules not processed for user `a800d4a53c234d2fa53eeb0c68163f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:24.584802 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-162ce5e2-1c54-483d-a222-48ef32c5c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:24.594531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1201/2395] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:25.277721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:25.278209 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:25.282012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:25.301621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] MFA Rules not processed for user `50d98bc599f74688b7f9fea92ee7c2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:25.303197 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:25.311895 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1195/2396] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:15:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:25.428877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:25.429196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:25.429445 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:25.469754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:15:25.472764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:25.524898 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:25.598511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1202/2397] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:15:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:15:25.616294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:25.619374 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:25.690405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:25.729329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aa422-fd10-4a7b-a9ee-abb04b235a5b None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:25.735393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:25.735955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:25.736270 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:25.736563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:25.748231 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:25.749369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:25.758472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1196/2398] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:15:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:15:35.638703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=17 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:35.810500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=16 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:15:57.438842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:57.440318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:57.452918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9558c-3e61-4c0b-ac35-4bb82c0fb610 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:57.457177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:57.457177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:57.457177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:57.457177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:57.465589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:57.524616 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:57.526028 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81911e-ffae-4088-a913-feae573940dd service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:57.535511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1203/2399] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:15:58.946870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:15:58.959750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dba1d-d86c-4c20-abd5-30beaf1c4de5 service swift] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:15:58.963519 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:15:58.964225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:15:58.964507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:15:58.964777 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:15:58.969383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:15:59.037487 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:15:59.038873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:15:59.048713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1197/2400] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:15:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:16:07.394202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:16:07.394775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:16:07.394775 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:16:07.414356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] MFA Rules not processed for user `d625c099d9134f78b3e8940879c19106`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:16:07.415558 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:16:07.422502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1204/2401] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:16:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:16:09.044733 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=28 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:16:17.426225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=33 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:16:31.051607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:16:31.061337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7792b-1593-42cb-acf0-53fd37386538 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:16:31.065095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:16:31.065704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:16:31.065984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:16:31.066259 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:16:31.068015 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:16:31.130306 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:16:31.131740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0520980-dc24-4da6-b89b-166896a50c79 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:16:31.141438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1198/2402] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:16:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:16:41.137120 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=30 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:16:44.016988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:16:44.018492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:16:44.106002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37c37a-7c24-400f-8d23-d7b77557e2b1 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:16:44.110940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:16:44.111561 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:16:44.111838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:16:44.112136 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:16:44.113825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:16:44.125455 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:16:44.127002 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f472143-a0b0-453f-ab9b-8f333575b2b2 service placement] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:16:44.136899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1205/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:16:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:16:54.132479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=41 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:16:55.863563 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce823d-ee77-4a52-9028-de54f15873c6 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:16:55.864162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce823d-ee77-4a52-9028-de54f15873c6 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:16:55.864162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce823d-ee77-4a52-9028-de54f15873c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:16:55.883615 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-41ce823d-ee77-4a52-9028-de54f15873c6 None None] MFA Rules not processed for user `a800d4a53c234d2fa53eeb0c68163f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:16:55.884832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-41ce823d-ee77-4a52-9028-de54f15873c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:16:55.891606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1199/2404] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:16:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:17:00.509798 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:17:00.510284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:17:00.510405 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:17:00.530173 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] MFA Rules not processed for user `50d98bc599f74688b7f9fea92ee7c2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:17:00.531367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:17:00.539460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1206/2405] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:17:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:17:05.878183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:17:10.524852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:17:59.596844 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:17:59.597415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:17:59.597415 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:17:59.617560 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:17:59.618798 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:17:59.632607 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1200/2406] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 10 17:17:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:17:59.672754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:17:59.684724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a848b1-75e0-4559-b8f4-9b3fd5a9facd None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:17:59.689698 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:17:59.690325 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:17:59.690604 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:17:59.690874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:17:59.701556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:17:59.704709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:17:59.714442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1207/2407] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:18:07.711141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:07.725516 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d16c21-ba30-4a03-a6d0-18aac1acc61e None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:07.732774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:07.733648 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:07.734107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:07.734567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] PATH_INFO: `/v3/users/a800d4a53c234d2fa53eeb0c68163f44` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:07.736189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a800d4a53c234d2fa53eeb0c68163f44)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:07.738639 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:07.943377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a800d4a53c234d2fa53eeb0c68163f44'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.133302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a800d4a53c234d2fa53eeb0c68163f44'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.133302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a800d4a53c234d2fa53eeb0c68163f44'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.133302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a800d4a53c234d2fa53eeb0c68163f44'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.133302 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a800d4a53c234d2fa53eeb0c68163f44'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.140368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1201/2408] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:18:07 2022] DELETE /identity/v3/users/a800d4a53c234d2fa53eeb0c68163f44 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:08.296685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:08.298299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:08.316878 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3242b3-2c9a-4071-aa1a-7f32ce203534 service swift] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:08.374123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:08.374825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.375105 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.375377 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] PATH_INFO: `/v3/users/e750af16ce0d4742a3305141470ff308` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.376530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e750af16ce0d4742a3305141470ff308)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:08.378432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:08.422769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:08.424244 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:08.438195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fbb0e-69e3-4685-84d1-48442be36730 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:08.442865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:08.443540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.443978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.444348 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] PATH_INFO: `/v3/projects/b9422ce68e164e7d8ae8dbf168e58e78` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.445480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9422ce68e164e7d8ae8dbf168e58e78)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:08.447362 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:08.619425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e750af16ce0d4742a3305141470ff308'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.623766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e750af16ce0d4742a3305141470ff308'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.624390 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9422ce68e164e7d8ae8dbf168e58e78', 'request_id': 'req-8e9211f5-e43c-4f05-874c-a138dcb63987'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.630398 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9422ce68e164e7d8ae8dbf168e58e78', 'request_id': 'req-8e9211f5-e43c-4f05-874c-a138dcb63987'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.637422 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9422ce68e164e7d8ae8dbf168e58e78', 'request_id': 'req-8e9211f5-e43c-4f05-874c-a138dcb63987'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.688752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e750af16ce0d4742a3305141470ff308'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.703238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e750af16ce0d4742a3305141470ff308'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.704099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-0d7510e8-447d-4067-8975-5e2fef8abe13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e750af16ce0d4742a3305141470ff308'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.717819 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1208/2409] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:18:08 2022] DELETE /identity/v3/users/e750af16ce0d4742a3305141470ff308 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:08.752445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] The token cache is being invalidate because project b9422ce68e164e7d8ae8dbf168e58e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:18:08.753000 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.753432 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-8e9211f5-e43c-4f05-874c-a138dcb63987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:08.755124 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1202/2410] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:18:08 2022] DELETE /identity/v3/projects/b9422ce68e164e7d8ae8dbf168e58e78 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:18:08.760766 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.761240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.761515 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.782030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:18:08.783280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:08.843282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1209/2411] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:18:08.851193 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.851423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.851704 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.872131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:18:08.873347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:08.881335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1203/2412] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:18:08.886534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:08.896264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763a9fd-f7d4-4edc-a6b8-157ee5be61f1 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:08.901084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:08.901672 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.901951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.902223 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.902874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:18:08.903178 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:08.904828 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:08.909205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:08.916730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1210/2413] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:18:08.921055 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:08.930201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a79e3-d9cb-4b40-b19b-d43940de240a None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:08.935482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:08.936127 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.936385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.936659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.937284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:08.939157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:08.961070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:08.965225 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:08.976611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96865af9-9276-43e5-8166-262b7519156e admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:08.981994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:08.982633 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:08.982916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:08.983219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] PATH_INFO: `/v3/projects/4f9731af8c4c4188bc61c7f35b8bdc5e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:08.984480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9731af8c4c4188bc61c7f35b8bdc5e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:08.987003 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.046037 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1204/2414] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:18:08 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:18:09.050096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.063736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:09.112884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb107cab-2943-42f0-b08a-c3b29aa0e210 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.117486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.118173 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.118454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.118722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.119334 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.121923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.151001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9731af8c4c4188bc61c7f35b8bdc5e', 'request_id': 'req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.219499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1205/2415] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:18:09 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:18:09.223538 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.233476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08314fa1-31d8-4633-80bc-cbe03a8720ff admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.242608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9731af8c4c4188bc61c7f35b8bdc5e', 'request_id': 'req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.251065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9731af8c4c4188bc61c7f35b8bdc5e', 'request_id': 'req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.264278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] The token cache is being invalidate because project 4f9731af8c4c4188bc61c7f35b8bdc5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:18:09.264568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.265245 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.266728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1211/2416] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:18:08 2022] DELETE /identity/v3/projects/4f9731af8c4c4188bc61c7f35b8bdc5e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:18:09.270608 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.271881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:09.296897 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.298109 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.298386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.298658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.299695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.301473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.312217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1206/2417] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:18:09.315789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.317474 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:09.329724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db506924-f267-4a0a-b0e3-6d8cea23c1e4 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.334420 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.335006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.335285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.335565 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.335726 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d48dd-f3c3-4360-a691-a841dcb0d0ca admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.336255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.337789 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.340615 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.341240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.341521 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.341797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] PATH_INFO: `/v3/users/6d1303f35bb24d56880116afa911467e` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.342943 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1303f35bb24d56880116afa911467e)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.344931 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.348387 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1207/2418] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:18:09.357430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.367382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a49d0-8485-4ea4-a29e-fe2b9223642e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.372769 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f4043-1104-4149-a279-9dcbe16f51ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.373404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f4043-1104-4149-a279-9dcbe16f51ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.373684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f4043-1104-4149-a279-9dcbe16f51ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.373956 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f4043-1104-4149-a279-9dcbe16f51ee admin admin] PATH_INFO: `/v3/projects/7eb5ff396cf04ec29557aaec897ba03f/users/2736a4cdd68a4f00871d2c4d7654605b/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.389632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f4043-1104-4149-a279-9dcbe16f51ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb5ff396cf04ec29557aaec897ba03f, user_id=2736a4cdd68a4f00871d2c4d7654605b, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.392301 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f4043-1104-4149-a279-9dcbe16f51ee admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.496477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1208/2419] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:18:09 2022] PUT /identity/v3/projects/7eb5ff396cf04ec29557aaec897ba03f/users/2736a4cdd68a4f00871d2c4d7654605b/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:18:09.507404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610545-248e-474a-afe0-bd3abc080339 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.507690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610545-248e-474a-afe0-bd3abc080339 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.507964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45610545-248e-474a-afe0-bd3abc080339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.527246 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1303f35bb24d56880116afa911467e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.527908 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1303f35bb24d56880116afa911467e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.536017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-45610545-248e-474a-afe0-bd3abc080339 None None] MFA Rules not processed for user `2736a4cdd68a4f00871d2c4d7654605b`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:18:09.537275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-45610545-248e-474a-afe0-bd3abc080339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:09.581328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1303f35bb24d56880116afa911467e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.590569 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:09.595602 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1303f35bb24d56880116afa911467e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.596309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1303f35bb24d56880116afa911467e'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:09.601255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1209/2420] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:18:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:18:09.605341 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:09.612399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1212/2421] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:18:09 2022] DELETE /identity/v3/users/6d1303f35bb24d56880116afa911467e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:18:09.612651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45610545-248e-474a-afe0-bd3abc080339 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.613865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-45610545-248e-474a-afe0-bd3abc080339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:09.671488 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45610545-248e-474a-afe0-bd3abc080339 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.722115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.722735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.723013 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.723286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.780312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.781845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:09.790740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1210/2422] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:18:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:18:09.884456 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:09.886553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:09.956808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd411e-eb03-4c8c-a767-4ef97483e469 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:09.961687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:09.962315 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:09.962689 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:09.962863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] PATH_INFO: `/v3/projects/dfaa60d75ad34d5dbe9402bb99667133` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:09.964603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfaa60d75ad34d5dbe9402bb99667133)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:09.966509 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:10.102141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfaa60d75ad34d5dbe9402bb99667133', 'request_id': 'req-2a302962-8188-4c5e-8ccd-20efb9eb4d48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:10.186995 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfaa60d75ad34d5dbe9402bb99667133', 'request_id': 'req-2a302962-8188-4c5e-8ccd-20efb9eb4d48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:10.193265 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfaa60d75ad34d5dbe9402bb99667133', 'request_id': 'req-2a302962-8188-4c5e-8ccd-20efb9eb4d48'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:10.205282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] The token cache is being invalidate because project dfaa60d75ad34d5dbe9402bb99667133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:18:10.205573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:10.206187 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:10.207669 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1213/2423] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:18:09 2022] DELETE /identity/v3/projects/dfaa60d75ad34d5dbe9402bb99667133 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:12.799437 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:12.800859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:12.811609 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c8619-073f-4f49-bbdf-bb1bfae5e1bd service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:12.860573 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:12.861113 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:12.861391 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:12.861665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:12.863475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:12.881843 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=100 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:12.944689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:12.946105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:12.967214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1211/2424] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:18:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:18:14.608387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:14.609771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:14.665619 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a302962-8188-4c5e-8ccd-20efb9eb4d48 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:14.669141 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:14.669730 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:14.670005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:14.670271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:14.671906 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:14.684704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:14.686100 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4522e-5a93-4b96-b1af-e3a39af89fe7 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:14.696642 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1214/2425] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:18:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:18:22.964676 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=34 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:24.691888 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:30.749972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:30.751357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:30.814128 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be16245-5b2a-4b39-b6ca-78349adef1cc service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:30.818585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:30.819202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:30.819478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:30.819748 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] PATH_INFO: `/v3/users/2736a4cdd68a4f00871d2c4d7654605b` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:30.820874 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2736a4cdd68a4f00871d2c4d7654605b)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:30.822742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:31.043114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2736a4cdd68a4f00871d2c4d7654605b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.095523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2736a4cdd68a4f00871d2c4d7654605b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.096181 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2736a4cdd68a4f00871d2c4d7654605b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.096809 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2736a4cdd68a4f00871d2c4d7654605b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.109811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2736a4cdd68a4f00871d2c4d7654605b'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.122175 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1212/2426] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:18:30 2022] DELETE /identity/v3/users/2736a4cdd68a4f00871d2c4d7654605b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:31.126611 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:31.127080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:31.127189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:31.146442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:18:31.147624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:31.204354 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1215/2427] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:18:31.213965 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:31.215931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:31.272453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d1266-1dc5-49ea-9142-92e1ac919a90 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:31.276121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93a052-938b-473e-ac77-eefffa1420f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:31.276724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93a052-938b-473e-ac77-eefffa1420f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:31.277006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93a052-938b-473e-ac77-eefffa1420f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:31.277278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93a052-938b-473e-ac77-eefffa1420f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:31.286077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93a052-938b-473e-ac77-eefffa1420f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:31.287505 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93a052-938b-473e-ac77-eefffa1420f9 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:31.296824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1213/2428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:18:31.719487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:31.721009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:31.734769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25f52c-7e9a-4aa1-8b43-5d6398ee2de2 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:31.740817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:31.741466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:31.741748 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:31.742079 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] PATH_INFO: `/v3/projects/7eb5ff396cf04ec29557aaec897ba03f` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:31.743184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb5ff396cf04ec29557aaec897ba03f)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:31.747871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:31.850207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb5ff396cf04ec29557aaec897ba03f', 'request_id': 'req-e7002f06-24fd-453b-8efd-72a12022d972'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.918764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb5ff396cf04ec29557aaec897ba03f', 'request_id': 'req-e7002f06-24fd-453b-8efd-72a12022d972'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.924847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb5ff396cf04ec29557aaec897ba03f', 'request_id': 'req-e7002f06-24fd-453b-8efd-72a12022d972'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.936911 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] The token cache is being invalidate because project 7eb5ff396cf04ec29557aaec897ba03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:18:31.937200 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.937825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:31.939314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1216/2429] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:18:31 2022] DELETE /identity/v3/projects/7eb5ff396cf04ec29557aaec897ba03f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:38.068788 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:38.069304 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:38.069655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:38.089917 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] MFA Rules not processed for user `50d98bc599f74688b7f9fea92ee7c2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:18:38.091117 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:38.108531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:38.147724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1214/2430] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:18:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:18:41.933149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:46.798418 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:46.799846 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:46.860401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7002f06-24fd-453b-8efd-72a12022d972 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:46.864426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:46.865040 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:46.865322 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:46.865597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:46.867366 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:46.932207 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:46.933643 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:46.943415 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1217/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:18:48.293760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=15 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:50.715889 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:50.726987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858333d7-f15e-4c1c-80c1-c99da5e74105 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:50.731358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:50.734992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:50.734992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:50.734992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:50.734992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:50.744775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:50.746465 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:50.759243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1215/2432] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:18:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:18:54.078626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:54.080880 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:54.097414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd63a6-eb25-4596-a13a-200cb63b8efb service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:54.099398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:54.100081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:54.100540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:54.100808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] PATH_INFO: `/v3/users/d625c099d9134f78b3e8940879c19106` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:54.101930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d625c099d9134f78b3e8940879c19106)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:54.103960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:54.272729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd625c099d9134f78b3e8940879c19106'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.273381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd625c099d9134f78b3e8940879c19106'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.338574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd625c099d9134f78b3e8940879c19106'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.351165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd625c099d9134f78b3e8940879c19106'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.351836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd625c099d9134f78b3e8940879c19106'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.364864 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1218/2433] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:18:54 2022] DELETE /identity/v3/users/d625c099d9134f78b3e8940879c19106 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:54.600245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:54.619808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f49aab-bfbe-490a-a888-8d345cea5086 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:54.693335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:54.693959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:54.694243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:54.695140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] PATH_INFO: `/v3/projects/c36e0d86e93942189945656ccb70c059` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:54.696372 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36e0d86e93942189945656ccb70c059)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:54.698271 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:54.816077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36e0d86e93942189945656ccb70c059', 'request_id': 'req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.821824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36e0d86e93942189945656ccb70c059', 'request_id': 'req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.829379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36e0d86e93942189945656ccb70c059', 'request_id': 'req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.895546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] The token cache is being invalidate because project c36e0d86e93942189945656ccb70c059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:18:54.895840 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.896499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:54.898202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1216/2434] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:18:54 2022] DELETE /identity/v3/projects/c36e0d86e93942189945656ccb70c059 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:54.902597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:54.904087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:54.916978 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad04182-f442-4dab-9c08-69482f5e7d82 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:54.935104 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:54.975122 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:54.975681 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:54.975965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:54.976269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] PATH_INFO: `/v3/users/74d1f362098343c7b99e2784e7944ed4` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:54.977464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d1f362098343c7b99e2784e7944ed4)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:54.979318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:55.180574 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d1f362098343c7b99e2784e7944ed4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.181252 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d1f362098343c7b99e2784e7944ed4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.264534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d1f362098343c7b99e2784e7944ed4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.278871 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d1f362098343c7b99e2784e7944ed4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.279641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d1f362098343c7b99e2784e7944ed4'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.293457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1219/2435] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:18:54 2022] DELETE /identity/v3/users/74d1f362098343c7b99e2784e7944ed4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:18:55.650545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:18:55.650545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:18:55.673408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef80ed-f9f4-4f06-b04e-85b4f31d5a00 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:18:55.749077 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:18:55.749716 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:18:55.750009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:18:55.750279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] PATH_INFO: `/v3/projects/769cd8462dd4456fa527c611463e4677` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:18:55.751408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=769cd8462dd4456fa527c611463e4677)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:18:55.753333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:18:55.775759 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:18:55.882468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769cd8462dd4456fa527c611463e4677', 'request_id': 'req-41f84c4e-fcd3-47eb-963f-ba4aac182a17'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.891552 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769cd8462dd4456fa527c611463e4677', 'request_id': 'req-41f84c4e-fcd3-47eb-963f-ba4aac182a17'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.898845 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769cd8462dd4456fa527c611463e4677', 'request_id': 'req-41f84c4e-fcd3-47eb-963f-ba4aac182a17'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.970699 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] The token cache is being invalidate because project 769cd8462dd4456fa527c611463e4677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:18:55.970989 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.971632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-41f84c4e-fcd3-47eb-963f-ba4aac182a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:18:55.973179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1217/2436] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:18:55 2022] DELETE /identity/v3/projects/769cd8462dd4456fa527c611463e4677 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:19:05.288674 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:19:06.133160 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:19:47.881400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:19:47.882668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:47.976495 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d6b9-1b58-40a2-b525-beb366d09c77 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:19:48.054610 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:19:48.055240 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:19:48.056107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:19:48.056107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:19:48.057684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:48.093701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:19:48.095883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:19:48.110809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1220/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:19:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:19:53.019888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:19:53.020435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:19:53.020554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:19:53.040327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] MFA Rules not processed for user `50d98bc599f74688b7f9fea92ee7c2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:19:53.041471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:53.098101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1218/2438] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:19:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:19:53.399426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:19:53.400835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:53.456570 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e380766-3022-4f1f-8447-d93fc56d8a86 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:19:53.460174 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:19:53.460781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:19:53.461060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:19:53.461333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:19:53.463148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:53.473277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:19:53.474682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:19:53.482627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1221/2439] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:19:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:19:54.893583 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:19:54.895003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:54.963808 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba615c-62d0-4611-99ef-38907b21d5ae None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:19:54.969700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:19:54.970359 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:19:54.970632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:19:54.970904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:19:54.972646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:19:54.985169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:19:54.986747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:19:54.997138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1219/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:19:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:20:03.478991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=88 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:20:04.992499 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=57 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:20:06.559361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:20:06.560836 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:20:06.577782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9361-0444-4c75-8de4-f347321356fd service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:20:06.583226 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:20:06.583824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:20:06.584133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:20:06.584410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:20:06.586056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:20:06.597848 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:20:06.599410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:20:06.609763 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1222/2441] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:20:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:20:16.605397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=6 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:20:47.012756 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:20:47.024961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87664d15-2797-4607-b2bb-965c7250d579 service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:20:47.030964 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:20:47.031556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:20:47.031831 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:20:47.032138 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:20:47.033822 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:20:47.043937 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:20:47.046176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:20:47.055008 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1220/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:20:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:20:57.050888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=6 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:09.432481 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:09.433894 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:09.445728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b11a2-6f09-46e1-ab04-be06c00e2136 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:09.449486 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:09.450081 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:09.450358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:09.450630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:09.452319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:09.513018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:09.514424 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:09.524362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1223/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:12.687096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:12.698531 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d599a-4648-4bca-95e2-02d42e9f7107 service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:12.704673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:12.705226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:12.705509 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:12.705774 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:12.707859 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:12.721279 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:12.722877 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:12.734048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1221/2444] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:12.839414 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:12.840957 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:12.853729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbc7e3-7bfc-42ce-9a84-eda91ea186fa service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:12.858447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:12.859086 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:12.859367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:12.859645 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] PATH_INFO: `/v3/users/50d98bc599f74688b7f9fea92ee7c2ea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:12.861387 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d98bc599f74688b7f9fea92ee7c2ea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:12.863309 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:13.035110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d98bc599f74688b7f9fea92ee7c2ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.035701 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d98bc599f74688b7f9fea92ee7c2ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.113042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d98bc599f74688b7f9fea92ee7c2ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.125621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d98bc599f74688b7f9fea92ee7c2ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.126273 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-4f61e651-196d-41d6-a1e0-b54febe3aa46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d98bc599f74688b7f9fea92ee7c2ea'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.138673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1224/2445] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:12 2022] DELETE /identity/v3/users/50d98bc599f74688b7f9fea92ee7c2ea => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:13.376556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:13.387320 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedb671-c169-429d-8f37-879825bcea56 service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:13.443004 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:13.443637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:13.443918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:13.444224 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] PATH_INFO: `/v3/projects/f4fa5392cfd049509a86b2d88d536409` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:13.445405 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fa5392cfd049509a86b2d88d536409)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:13.447276 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:13.551980 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fa5392cfd049509a86b2d88d536409', 'request_id': 'req-56adb707-6755-4d07-97c3-3915e1a8cdca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.557545 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fa5392cfd049509a86b2d88d536409', 'request_id': 'req-56adb707-6755-4d07-97c3-3915e1a8cdca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.563647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fa5392cfd049509a86b2d88d536409', 'request_id': 'req-56adb707-6755-4d07-97c3-3915e1a8cdca'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.661411 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] The token cache is being invalidate because project f4fa5392cfd049509a86b2d88d536409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:13.661637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.662255 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-56adb707-6755-4d07-97c3-3915e1a8cdca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:13.663762 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1222/2446] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:13 2022] DELETE /identity/v3/projects/f4fa5392cfd049509a86b2d88d536409 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:21:13.668630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:13.668919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:13.669196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:13.689014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:13.690237 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:13.747663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1225/2447] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:13.755619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:13.756115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:13.756237 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:13.775546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:13.776778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:13.785177 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1223/2448] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:13.789107 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:13.799218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0604d7-7930-44ec-9aa0-bc001dbecbcf None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:13.803729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:13.804342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:13.804623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:13.804893 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:13.805530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:21:13.805829 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:13.807391 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:13.808831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:13.819063 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1226/2449] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:21:13.823212 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:13.833208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56881a6-e2e5-4f9a-b575-3d87bb2371ef None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:13.837898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:13.838489 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:13.838786 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:13.839043 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:13.839673 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:13.841570 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:13.915522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1224/2450] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:13 2022] POST /identity/v3/projects => generated 403 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:21:13.919511 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:13.929677 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b626308-1066-4a39-ba68-dbfba1dcf1cc admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:13.985831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:13.986428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:13.986709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:13.986983 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:13.987614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:13.989600 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.090236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1227/2451] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:13 2022] POST /identity/v3/users => generated 302 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:21:14.094275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.105257 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce710e-acb8-4d5c-81a8-b7ab11e8f646 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.109884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.111689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.111689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.111689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.111689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.113397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.123132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1225/2452] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:14.127150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.137356 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17edf65-ba13-4ef2-827d-b1df2a24e5d5 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.142557 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f4f3c-e1a4-4988-a890-c70f737d6937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.143148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f4f3c-e1a4-4988-a890-c70f737d6937 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.143476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f4f3c-e1a4-4988-a890-c70f737d6937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.143752 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f4f3c-e1a4-4988-a890-c70f737d6937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.144412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f4f3c-e1a4-4988-a890-c70f737d6937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.145991 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f4f3c-e1a4-4988-a890-c70f737d6937 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.155601 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1228/2453] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:14.160202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.170161 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fac4c-ef51-4478-a076-77fc0a5522c5 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.174712 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.175351 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.175629 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.175927 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] PATH_INFO: `/v3/projects/c4dccd748ade4c94ab254c32dc313306/users/e9be344ee55d4da694e4703e968c5c6a/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.190679 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4dccd748ade4c94ab254c32dc313306, user_id=e9be344ee55d4da694e4703e968c5c6a, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.193316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.256012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1226/2454] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:14 2022] PUT /identity/v3/projects/c4dccd748ade4c94ab254c32dc313306/users/e9be344ee55d4da694e4703e968c5c6a/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:14.268097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.268381 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.268651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.297206 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] MFA Rules not processed for user `e9be344ee55d4da694e4703e968c5c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:14.298429 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:14.360947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1229/2455] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:14 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:14.365518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.379081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:14.426385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0b532-5121-4aec-9d86-d620be3f0d33 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.431632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.432323 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.432602 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.432871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.433490 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.435357 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.506886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1227/2456] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:14 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:14.510963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.522087 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96175ca4-b47c-4fb4-8e98-65e77502ebdd None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.576250 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.576847 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.577123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.577392 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.578279 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.580229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.593480 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:14.673229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1230/2457] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:14 2022] POST /identity/v3/users => generated 309 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:14.679340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.687806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d920585-1ea6-4160-93a5-1a683b2eac94 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.693108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.693696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.693976 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.694248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.694887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.696491 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.707399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1228/2458] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:14.711477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.721471 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533e6ce-ac55-4066-9b63-6548efd90330 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.726069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.726712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.726996 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.727271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] PATH_INFO: `/v3/projects/e204a626cffb4a14b6b0e7da79a7eb56/users/819ddef2948a47fb9556d85184d6e9ee/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.742159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e204a626cffb4a14b6b0e7da79a7eb56, user_id=819ddef2948a47fb9556d85184d6e9ee, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.744808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.798851 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1231/2459] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:14 2022] PUT /identity/v3/projects/e204a626cffb4a14b6b0e7da79a7eb56/users/819ddef2948a47fb9556d85184d6e9ee/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:14.802862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.814009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425eebb-91bc-43e6-90b7-7b66d6122171 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.871723 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e8b21-0e33-402b-a108-8912f8ae1942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.872479 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8b21-0e33-402b-a108-8912f8ae1942 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.872755 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8b21-0e33-402b-a108-8912f8ae1942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.873021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e8b21-0e33-402b-a108-8912f8ae1942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.873649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e8b21-0e33-402b-a108-8912f8ae1942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.875247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e8b21-0e33-402b-a108-8912f8ae1942 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.885682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1229/2460] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:14.889905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:14.900201 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5c9b-25ca-42b8-849a-349ac57758d3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:14.904782 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:14.905428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.905710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.905979 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] PATH_INFO: `/v3/projects/e204a626cffb4a14b6b0e7da79a7eb56/users/819ddef2948a47fb9556d85184d6e9ee/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.908505 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e204a626cffb4a14b6b0e7da79a7eb56, user_id=819ddef2948a47fb9556d85184d6e9ee, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:14.911099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:14.921589 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1232/2461] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:14 2022] PUT /identity/v3/projects/e204a626cffb4a14b6b0e7da79a7eb56/users/819ddef2948a47fb9556d85184d6e9ee/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:21:14.932518 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:14.932740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:14.933012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:14.960423 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] MFA Rules not processed for user `819ddef2948a47fb9556d85184d6e9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:14.961627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:15.028047 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1230/2462] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:14 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:15.039417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:15.041171 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:15.098159 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9874de9-891d-4457-a928-cc449490d838 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:15.101834 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055a8a0-c881-48d2-a7b7-2f237de675d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:15.102432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055a8a0-c881-48d2-a7b7-2f237de675d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:15.102712 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055a8a0-c881-48d2-a7b7-2f237de675d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:15.102984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055a8a0-c881-48d2-a7b7-2f237de675d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:15.112737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055a8a0-c881-48d2-a7b7-2f237de675d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:15.114233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055a8a0-c881-48d2-a7b7-2f237de675d1 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:15.123939 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1233/2463] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:15 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:15.283832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:15.293368 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:15.345824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ac80-c2c4-452d-b6b3-344a44183c84 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:15.350413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:15.351037 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:15.351322 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:15.351598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] PATH_INFO: `/v3/users/e9be344ee55d4da694e4703e968c5c6a` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:15.352952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9be344ee55d4da694e4703e968c5c6a)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:15.355164 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:15.520354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9be344ee55d4da694e4703e968c5c6a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:15.575623 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9be344ee55d4da694e4703e968c5c6a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:15.576252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9be344ee55d4da694e4703e968c5c6a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:15.576857 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9be344ee55d4da694e4703e968c5c6a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:15.589051 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9be344ee55d4da694e4703e968c5c6a'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:15.601132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1231/2464] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:15 2022] DELETE /identity/v3/users/e9be344ee55d4da694e4703e968c5c6a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:15.605743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:15.606031 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:15.606308 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:15.625291 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:15.627123 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:15.685258 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1234/2465] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:15.695497 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:15.698281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:15.754932 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af52e9-a675-44b0-8ab3-69dc119e30eb admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:15.758595 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:15.759195 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:15.759472 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:15.759740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:15.769156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:15.770619 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:15.779794 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1232/2466] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:16.173448 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:16.179065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:16.186540 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba944-ce3a-4fac-b461-b2d9022cb66f None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:16.191464 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:16.192149 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:16.192490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:16.192760 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] PATH_INFO: `/v3/projects/c4dccd748ade4c94ab254c32dc313306` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:16.194443 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4dccd748ade4c94ab254c32dc313306)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:16.196565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:16.302011 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4dccd748ade4c94ab254c32dc313306', 'request_id': 'req-1c8ecd8c-dad8-4a09-8d94-379a13e11933'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.304426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:16.404649 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4dccd748ade4c94ab254c32dc313306', 'request_id': 'req-1c8ecd8c-dad8-4a09-8d94-379a13e11933'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.414484 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4dccd748ade4c94ab254c32dc313306', 'request_id': 'req-1c8ecd8c-dad8-4a09-8d94-379a13e11933'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.426818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] The token cache is being invalidate because project c4dccd748ade4c94ab254c32dc313306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:16.427108 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.427744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.429234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1235/2467] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:16 2022] DELETE /identity/v3/projects/c4dccd748ade4c94ab254c32dc313306 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:21:16.433261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:16.442931 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14501baa-d146-4213-86e2-b3f913cb6913 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:16.472484 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:16.499767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:16.500434 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:16.500713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:16.500986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] PATH_INFO: `/v3/users/819ddef2948a47fb9556d85184d6e9ee` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:16.502093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=819ddef2948a47fb9556d85184d6e9ee)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:16.503982 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:16.664632 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '819ddef2948a47fb9556d85184d6e9ee'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.713337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '819ddef2948a47fb9556d85184d6e9ee'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.713959 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '819ddef2948a47fb9556d85184d6e9ee'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.714559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '819ddef2948a47fb9556d85184d6e9ee'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.727021 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-55f63790-fcaf-4ede-ae39-be6d60f08c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '819ddef2948a47fb9556d85184d6e9ee'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:16.739767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1233/2468] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:16 2022] DELETE /identity/v3/users/819ddef2948a47fb9556d85184d6e9ee => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:17.108467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.109874 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:17.122704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ecd8c-dad8-4a09-8d94-379a13e11933 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.179165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.179804 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.179972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.180269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] PATH_INFO: `/v3/projects/e204a626cffb4a14b6b0e7da79a7eb56` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.181360 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e204a626cffb4a14b6b0e7da79a7eb56)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.183215 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.306617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e204a626cffb4a14b6b0e7da79a7eb56', 'request_id': 'req-8ca637e0-6371-4d62-accf-8731d6951d30'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:17.364085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e204a626cffb4a14b6b0e7da79a7eb56', 'request_id': 'req-8ca637e0-6371-4d62-accf-8731d6951d30'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:17.369722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e204a626cffb4a14b6b0e7da79a7eb56', 'request_id': 'req-8ca637e0-6371-4d62-accf-8731d6951d30'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:17.381298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] The token cache is being invalidate because project e204a626cffb4a14b6b0e7da79a7eb56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:17.382925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:17.382925 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-8ca637e0-6371-4d62-accf-8731d6951d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:17.383683 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1236/2469] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:17 2022] DELETE /identity/v3/projects/e204a626cffb4a14b6b0e7da79a7eb56 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:17.388637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.388920 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.389189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.409214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:17.410408 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:17.468803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1234/2470] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:17.476568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.476858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.477140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.496738 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:17.497920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:17.506342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1237/2471] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:17.510486 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.520655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6387ca-9214-449f-8ff5-9583cbc7fa5a None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.525253 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.525900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.526183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.526459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.527104 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:21:17.527407 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.529003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.540316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1235/2472] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:21:17.544500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.554069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62ea44-2f74-4eac-b74f-a8d08e085e9a None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.558725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.559316 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.559594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.559870 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.560535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.562406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.649413 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1238/2473] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:17 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:17.653456 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.663229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7e058-dfa2-4129-b0c2-5ed212ad9b1f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.696438 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:17.719384 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.719935 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.720245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.720517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.721141 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.723018 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.814827 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1236/2474] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:17 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:17.818958 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.829618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d74871-72fd-459c-a189-86c6b3010f5c admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.834412 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.835004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.835278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.835547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.836203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.837808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.847512 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1239/2475] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:17.851298 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.861760 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7575f62-205a-4865-bf71-a2685b9d9441 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.866385 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be9740-1b32-4d46-b0ce-7c26e2e413fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.866984 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be9740-1b32-4d46-b0ce-7c26e2e413fb admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.867266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be9740-1b32-4d46-b0ce-7c26e2e413fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.867537 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be9740-1b32-4d46-b0ce-7c26e2e413fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.868214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be9740-1b32-4d46-b0ce-7c26e2e413fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.869801 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be9740-1b32-4d46-b0ce-7c26e2e413fb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.880547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1237/2476] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:17.884575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:17.894459 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999bda0-abbb-4a83-8e39-919d69177dc0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:17.899028 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:17.899818 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.900129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.900402 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] PATH_INFO: `/v3/projects/45ab0b48ec934775a923b416a863630b/users/4d02479e978f4ab192cc7228a395657d/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:17.915344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ab0b48ec934775a923b416a863630b, user_id=4d02479e978f4ab192cc7228a395657d, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:17.917993 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:17.987740 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1240/2477] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:17 2022] PUT /identity/v3/projects/45ab0b48ec934775a923b416a863630b/users/4d02479e978f4ab192cc7228a395657d/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:17.999038 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:17.999335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:17.999618 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.029442 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] MFA Rules not processed for user `4d02479e978f4ab192cc7228a395657d`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:18.030967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:18.093402 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1238/2478] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:18.097886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.106869 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:18.159436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549cd7b-c753-4bcc-b57c-335570739d65 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.164129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.164737 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.165022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.165295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.165937 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.167905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.256789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1241/2479] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:18 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:21:18.260945 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.272227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501b86d-df8f-4782-8102-8c211b2b6c96 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.327447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.328339 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.328622 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.328913 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.329561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.331453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.398772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1239/2480] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:18 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:18.405647 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.415886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153eb336-06e1-406a-b83c-577f91043e94 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.420685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.421282 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.421560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.421831 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.422463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.424568 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.435195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1242/2481] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:18.440085 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.450266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba73a1-c787-43ae-97de-c57f9a8324ba admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.455294 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.455943 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.456247 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.456522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] PATH_INFO: `/v3/projects/d67c9b9734f74ad19e9cd166c38d4ff3/users/7eeeae24522b453b961b1c3c2fcce665/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.471012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67c9b9734f74ad19e9cd166c38d4ff3, user_id=7eeeae24522b453b961b1c3c2fcce665, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.475155 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.570103 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1240/2482] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:18 2022] PUT /identity/v3/projects/d67c9b9734f74ad19e9cd166c38d4ff3/users/7eeeae24522b453b961b1c3c2fcce665/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:18.573867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.583973 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229141-5e6c-4330-997d-9b6f30cbeb37 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.640781 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e1b87-db84-481d-9cbf-ee4e8d56b2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.641401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e1b87-db84-481d-9cbf-ee4e8d56b2fd admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.641684 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e1b87-db84-481d-9cbf-ee4e8d56b2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.641956 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e1b87-db84-481d-9cbf-ee4e8d56b2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.642594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e1b87-db84-481d-9cbf-ee4e8d56b2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.644243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e1b87-db84-481d-9cbf-ee4e8d56b2fd admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.653940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1243/2483] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:18.658551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.668665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74a6c-ac22-4647-bc54-a5426d8da8a2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.673908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.674554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.674837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.675137 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] PATH_INFO: `/v3/projects/d67c9b9734f74ad19e9cd166c38d4ff3/users/7eeeae24522b453b961b1c3c2fcce665/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.677094 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67c9b9734f74ad19e9cd166c38d4ff3, user_id=7eeeae24522b453b961b1c3c2fcce665, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.679685 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.685067 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:18.690130 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1241/2484] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:18 2022] PUT /identity/v3/projects/d67c9b9734f74ad19e9cd166c38d4ff3/users/7eeeae24522b453b961b1c3c2fcce665/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:18.711592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.711592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.711592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.728582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] MFA Rules not processed for user `7eeeae24522b453b961b1c3c2fcce665`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:18.729786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:18.796144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1244/2485] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:21:18.807900 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:18.809598 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:18.865752 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee188bd-9304-4bf9-9d46-56e14923d93f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:18.869396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:18.870003 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:18.870281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:18.870554 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:18.879575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:18.881084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:18.890207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1242/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:19.073941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:19.078791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:19.084247 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca866f-b644-4b3a-a01a-88ba948a3964 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:19.087839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b2560-05f6-49fe-b3f2-c8fe740297dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:19.088463 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b2560-05f6-49fe-b3f2-c8fe740297dd service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:19.088745 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b2560-05f6-49fe-b3f2-c8fe740297dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:19.089018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b2560-05f6-49fe-b3f2-c8fe740297dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:19.145556 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b2560-05f6-49fe-b3f2-c8fe740297dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:19.146873 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b2560-05f6-49fe-b3f2-c8fe740297dd service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:19.155503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1245/2487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:22.596266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:22.656700 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f23fb-d939-415e-9b31-57a9a300e491 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:22.661736 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:22.662356 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:22.662634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:22.662908 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] PATH_INFO: `/v3/users/4d02479e978f4ab192cc7228a395657d` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:22.664012 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d02479e978f4ab192cc7228a395657d)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:22.665953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:22.816453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d02479e978f4ab192cc7228a395657d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:22.894717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d02479e978f4ab192cc7228a395657d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:22.895337 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d02479e978f4ab192cc7228a395657d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:22.895971 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d02479e978f4ab192cc7228a395657d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:22.909073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d02479e978f4ab192cc7228a395657d'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:22.921349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1243/2488] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:22 2022] DELETE /identity/v3/users/4d02479e978f4ab192cc7228a395657d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:22.925736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:22.926219 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:22.926324 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:22.944896 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:22.946077 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:23.003067 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1246/2489] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:23.012721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:23.014179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:23.036709 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:23.070735 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be2c3d-ef0e-4fef-a81a-4c446b2d4aa9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:23.074370 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:23.074973 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:23.075250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:23.075520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:23.085101 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:23.086508 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:23.096668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1244/2490] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:23.474026 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:23.475425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:23.490919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef57427-0e73-4b95-860f-1193667e8717 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:23.495805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:23.496417 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:23.500046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:23.500046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] PATH_INFO: `/v3/projects/45ab0b48ec934775a923b416a863630b` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:23.500046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ab0b48ec934775a923b416a863630b)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:23.500046 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:23.597770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ab0b48ec934775a923b416a863630b', 'request_id': 'req-d5fae683-a1c1-4a47-9337-aeaecbacb398'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:23.665916 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ab0b48ec934775a923b416a863630b', 'request_id': 'req-d5fae683-a1c1-4a47-9337-aeaecbacb398'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:23.672006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ab0b48ec934775a923b416a863630b', 'request_id': 'req-d5fae683-a1c1-4a47-9337-aeaecbacb398'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:23.683641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] The token cache is being invalidate because project 45ab0b48ec934775a923b416a863630b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:23.683922 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:23.686592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:23.686592 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1247/2491] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:23 2022] DELETE /identity/v3/projects/45ab0b48ec934775a923b416a863630b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:23.690722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:23.700429 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1c924-f330-4205-acd5-464c55d0a36d service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:23.756166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:23.756726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:23.757009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:23.757284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] PATH_INFO: `/v3/users/7eeeae24522b453b961b1c3c2fcce665` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:23.758485 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eeeae24522b453b961b1c3c2fcce665)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:23.760454 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:23.942258 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eeeae24522b453b961b1c3c2fcce665'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.003183 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eeeae24522b453b961b1c3c2fcce665'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.003807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eeeae24522b453b961b1c3c2fcce665'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.004433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eeeae24522b453b961b1c3c2fcce665'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.021614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-07170334-23aa-4ade-9acb-8a84b9de76cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eeeae24522b453b961b1c3c2fcce665'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.029229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1245/2492] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:23 2022] DELETE /identity/v3/users/7eeeae24522b453b961b1c3c2fcce665 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:24.439408 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:24.440940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:24.455001 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fae683-a1c1-4a47-9337-aeaecbacb398 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:24.475883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:24.514492 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:24.515124 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:24.515403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:24.515853 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] PATH_INFO: `/v3/projects/d67c9b9734f74ad19e9cd166c38d4ff3` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:24.517442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d67c9b9734f74ad19e9cd166c38d4ff3)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:24.519482 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:24.609467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd67c9b9734f74ad19e9cd166c38d4ff3', 'request_id': 'req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.677678 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd67c9b9734f74ad19e9cd166c38d4ff3', 'request_id': 'req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.683722 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd67c9b9734f74ad19e9cd166c38d4ff3', 'request_id': 'req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.695339 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] The token cache is being invalidate because project d67c9b9734f74ad19e9cd166c38d4ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:24.695624 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.696287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-97bd7ad7-8f4d-4ac4-874f-19f6dc116a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:24.697761 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1248/2493] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:24 2022] DELETE /identity/v3/projects/d67c9b9734f74ad19e9cd166c38d4ff3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:24.702645 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:24.703382 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:24.703483 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:24.723097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:24.724330 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:24.762425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:24.782717 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1246/2494] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:21:24.845288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:24.845566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:24.845838 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:24.869278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:24.869278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:24.878706 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1249/2495] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:24.882904 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:24.893312 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22612244-6f46-467b-ba99-9a6d4b6b1c23 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:24.898517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:24.899116 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:24.899392 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:24.899658 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:24.900394 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:21:24.900744 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:24.902335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:24.914397 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1247/2496] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:21:24.918581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:24.928009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbf7d2-f3ca-4a91-81b6-9e156270ad84 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:24.933653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:24.933653 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:24.933867 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:24.933984 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:24.934637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:24.936531 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.034137 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1250/2497] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:24 2022] POST /identity/v3/projects => generated 407 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:25.038275 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.048816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47b0a-47e2-44ee-84bf-51a437d7900d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.105670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.106272 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.106549 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.106819 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.107439 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.109344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.198815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1248/2498] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:25 2022] POST /identity/v3/users => generated 304 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:25.202887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.213470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b3f0e-b85b-44ae-a04e-2075362bab31 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.218271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.218881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.219165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.219439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.220133 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.221741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.231618 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1251/2499] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:25.235863 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.246119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777dc9d-a8fa-49ba-96d6-bd931c5336cb admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.251540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc391c0-2078-453b-969b-b3ac07b27b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.252189 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc391c0-2078-453b-969b-b3ac07b27b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.252462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc391c0-2078-453b-969b-b3ac07b27b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.252724 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc391c0-2078-453b-969b-b3ac07b27b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.253350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc391c0-2078-453b-969b-b3ac07b27b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.254901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc391c0-2078-453b-969b-b3ac07b27b07 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.265317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1249/2500] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:21:25.269364 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.279369 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92802bda-36d1-4ee2-9279-cf5f2fc0df88 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.284016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.284685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.284965 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.285242 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] PATH_INFO: `/v3/projects/90c926239db24d6a81ca3f7497bae539/users/aeb7d0694dc44ae09515ea2955d7298f/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.299902 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c926239db24d6a81ca3f7497bae539, user_id=aeb7d0694dc44ae09515ea2955d7298f, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.302581 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.352084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1252/2501] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:25 2022] PUT /identity/v3/projects/90c926239db24d6a81ca3f7497bae539/users/aeb7d0694dc44ae09515ea2955d7298f/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:25.362740 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.363112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.363453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.391119 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] MFA Rules not processed for user `aeb7d0694dc44ae09515ea2955d7298f`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:25.392413 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:25.454940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1250/2502] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:25.459426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.510276 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:25.521805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087450e8-3e60-4b38-9747-05c0866231fe admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.526542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.527142 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.527432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.527725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.528397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.530277 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.626940 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1253/2503] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:25 2022] POST /identity/v3/projects => generated 409 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:25.630998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.642241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba4543-f611-44b8-ba66-6402f498fcad None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.674369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:25.697750 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.698386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.698666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.698938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.699574 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.701480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.803803 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1251/2504] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:25 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:21:25.807765 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.819628 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bf126-9989-4a65-8588-de9eb8153ea6 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.824295 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.824926 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.825202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.825474 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.826431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.828285 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.838406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1254/2505] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:25.842401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.852100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf7851-0d70-4abb-917d-12d2b787bcf9 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:25.856651 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:25.857285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:25.857567 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:25.857837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] PATH_INFO: `/v3/projects/a59d3bebf0884183a9242186a386b5d6/users/48d4f8e8d9b54bae944f0a53ccba5a47/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:25.873467 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59d3bebf0884183a9242186a386b5d6, user_id=48d4f8e8d9b54bae944f0a53ccba5a47, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:25.876761 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:25.935584 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1252/2506] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:25 2022] PUT /identity/v3/projects/a59d3bebf0884183a9242186a386b5d6/users/48d4f8e8d9b54bae944f0a53ccba5a47/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:21:25.940376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:25.950362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668a303-2c1a-4b2a-bf93-793414151354 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:26.005564 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ddac7-e90d-405a-a3b0-77d55bdd75dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:26.006162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ddac7-e90d-405a-a3b0-77d55bdd75dc admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:26.006442 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ddac7-e90d-405a-a3b0-77d55bdd75dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:26.006717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ddac7-e90d-405a-a3b0-77d55bdd75dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:26.007342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ddac7-e90d-405a-a3b0-77d55bdd75dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:26.008947 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ddac7-e90d-405a-a3b0-77d55bdd75dc admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:26.018421 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1255/2507] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:26.022637 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:26.032856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e7e1-cbae-406a-bf9f-69b5668e682e admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:26.037473 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:26.038114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:26.038399 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:26.038671 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] PATH_INFO: `/v3/projects/a59d3bebf0884183a9242186a386b5d6/users/48d4f8e8d9b54bae944f0a53ccba5a47/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:26.040605 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59d3bebf0884183a9242186a386b5d6, user_id=48d4f8e8d9b54bae944f0a53ccba5a47, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:26.043196 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:26.053718 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1253/2508] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:26 2022] PUT /identity/v3/projects/a59d3bebf0884183a9242186a386b5d6/users/48d4f8e8d9b54bae944f0a53ccba5a47/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:26.064338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:26.064587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:26.064863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:26.092863 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] MFA Rules not processed for user `48d4f8e8d9b54bae944f0a53ccba5a47`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:26.094060 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:26.160437 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1256/2509] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:26 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:21:26.171022 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:26.173347 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:26.229329 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57c1a4-486e-42a6-aa8a-8fab391bea3d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:26.232962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:26.233553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:26.233832 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:26.234100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:26.243419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:26.244862 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:26.254179 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1254/2510] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:26 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:26.632710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:26.642858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682fb8e-e455-4cfa-968d-7c471187caef None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:26.646436 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f94b6-6cb0-42e3-8530-983720e1dce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:26.647032 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f94b6-6cb0-42e3-8530-983720e1dce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:26.647313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f94b6-6cb0-42e3-8530-983720e1dce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:26.647582 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f94b6-6cb0-42e3-8530-983720e1dce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:26.691580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:26.704595 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f94b6-6cb0-42e3-8530-983720e1dce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:26.705999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f94b6-6cb0-42e3-8530-983720e1dce6 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:26.714839 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1257/2511] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:26 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:36.249888 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=53,keystone:INSERT=5 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:36.555571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:36.615936 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949b7fc-45c4-4a04-89f4-91673f3bf2b0 service cinder] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:36.620424 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:36.621044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:36.621324 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:36.621597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] PATH_INFO: `/v3/users/aeb7d0694dc44ae09515ea2955d7298f` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:36.622726 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb7d0694dc44ae09515ea2955d7298f)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:36.624659 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:36.751963 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=7 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:36.823723 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb7d0694dc44ae09515ea2955d7298f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:36.895355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb7d0694dc44ae09515ea2955d7298f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:36.895972 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb7d0694dc44ae09515ea2955d7298f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:36.896614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb7d0694dc44ae09515ea2955d7298f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:36.909295 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb7d0694dc44ae09515ea2955d7298f'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:36.921564 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1255/2512] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:36 2022] DELETE /identity/v3/users/aeb7d0694dc44ae09515ea2955d7298f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:36.926420 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:36.926703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:36.926972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:36.947229 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:36.948452 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:37.005319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1258/2513] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:37.014837 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:37.016284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:37.072575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751e5eb-b27e-4dc5-bc97-21473439904f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:37.076225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:37.076770 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:37.077048 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:37.077321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:37.086342 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:37.087738 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:37.096995 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1256/2514] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:37.496000 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:37.497453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:37.514614 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5757a1d-e665-4ad6-894f-61c40fdc2671 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:37.519462 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:37.520158 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:37.520440 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:37.520771 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] PATH_INFO: `/v3/projects/90c926239db24d6a81ca3f7497bae539` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:37.521827 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c926239db24d6a81ca3f7497bae539)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:37.523742 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:37.642428 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c926239db24d6a81ca3f7497bae539', 'request_id': 'req-2b5524cb-0593-404c-8e99-3a2b986d8a2f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:37.711279 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c926239db24d6a81ca3f7497bae539', 'request_id': 'req-2b5524cb-0593-404c-8e99-3a2b986d8a2f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:37.716959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c926239db24d6a81ca3f7497bae539', 'request_id': 'req-2b5524cb-0593-404c-8e99-3a2b986d8a2f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:37.728433 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] The token cache is being invalidate because project 90c926239db24d6a81ca3f7497bae539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:37.728728 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:37.729383 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:37.730877 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1259/2515] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:37 2022] DELETE /identity/v3/projects/90c926239db24d6a81ca3f7497bae539 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:37.734966 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:37.745492 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85cd3a-9fa3-40b9-871d-88efd6884376 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:37.763400 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:37.803133 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:37.803778 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:37.804079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:37.804350 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] PATH_INFO: `/v3/users/48d4f8e8d9b54bae944f0a53ccba5a47` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:37.805445 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d4f8e8d9b54bae944f0a53ccba5a47)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:37.807369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:37.994482 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d4f8e8d9b54bae944f0a53ccba5a47'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.088523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d4f8e8d9b54bae944f0a53ccba5a47'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.089168 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d4f8e8d9b54bae944f0a53ccba5a47'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.089779 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d4f8e8d9b54bae944f0a53ccba5a47'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.102553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-33a1e2b3-87a5-47b8-8e81-d84404fbb46c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d4f8e8d9b54bae944f0a53ccba5a47'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.114646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1257/2516] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:21:37 2022] DELETE /identity/v3/users/48d4f8e8d9b54bae944f0a53ccba5a47 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:38.530503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:38.531959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:38.545075 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5524cb-0593-404c-8e99-3a2b986d8a2f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:38.601536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:38.602166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:38.602441 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:38.602710 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] PATH_INFO: `/v3/projects/a59d3bebf0884183a9242186a386b5d6` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:38.603800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59d3bebf0884183a9242186a386b5d6)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:38.605704 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:38.692139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:38.708185 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59d3bebf0884183a9242186a386b5d6', 'request_id': 'req-7583955a-30dd-439e-9ab1-53e2c41eed49'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.775234 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59d3bebf0884183a9242186a386b5d6', 'request_id': 'req-7583955a-30dd-439e-9ab1-53e2c41eed49'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.781106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59d3bebf0884183a9242186a386b5d6', 'request_id': 'req-7583955a-30dd-439e-9ab1-53e2c41eed49'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.792097 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] The token cache is being invalidate because project a59d3bebf0884183a9242186a386b5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:21:38.792390 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.793041 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-7583955a-30dd-439e-9ab1-53e2c41eed49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:21:38.794534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1260/2517] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:21:38 2022] DELETE /identity/v3/projects/a59d3bebf0884183a9242186a386b5d6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:38.800001 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:38.800477 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:38.800593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:38.820421 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:38.821635 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:38.880517 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1258/2518] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:21:38.888165 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:38.888453 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:38.888729 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:38.908777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:38.909988 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:38.917972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1261/2519] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:38.922042 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:38.932108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9a101-8c0a-4493-8a7b-25a93eba5666 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:38.936559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:38.937144 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:38.937419 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:38.937687 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:38.938316 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:21:38.938617 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:38.940269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:38.951674 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1259/2520] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:21:38.955974 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:38.965057 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59e175-c1fb-4812-bf89-48dc97340b5c None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:38.969762 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:38.970889 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:38.971166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:38.971435 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:38.972166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:38.974082 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:39.062313 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1262/2521] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:21:38 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:39.066841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:39.076816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02dbb7-b692-4330-8015-225b0b02b17f admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:39.102828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:39.132614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:39.133203 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:39.133480 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:39.133749 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:39.134460 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:39.136378 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:39.214333 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1260/2522] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:21:39 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:21:39.218502 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:39.229485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a26bea-f904-4d51-bdc8-58813c5a63f1 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:39.234062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:39.234691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:39.234970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:39.235243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:39.236626 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:39.238217 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:39.248202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1263/2523] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:39.252204 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:39.262069 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bc93-7083-4f1f-8748-9f58de0b05b0 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:39.266691 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec97a4-4c82-4c7f-9bc7-d0cc9f291c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:39.267785 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec97a4-4c82-4c7f-9bc7-d0cc9f291c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:39.268097 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec97a4-4c82-4c7f-9bc7-d0cc9f291c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:39.268379 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec97a4-4c82-4c7f-9bc7-d0cc9f291c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:39.269014 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec97a4-4c82-4c7f-9bc7-d0cc9f291c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:39.270585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec97a4-4c82-4c7f-9bc7-d0cc9f291c77 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:39.281056 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1261/2524] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:21:39.285687 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:39.295447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea2c86-49fc-49a2-8d90-d25c639860dd admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:39.299986 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:39.300651 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:39.300930 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:39.301203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] PATH_INFO: `/v3/projects/9888d15cc64842d397087e5c5265ca64/users/9b63724faba746b5b4df8bba23c08948/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:39.315202 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9888d15cc64842d397087e5c5265ca64, user_id=9b63724faba746b5b4df8bba23c08948, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:39.317791 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:39.373890 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1264/2525] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:21:39 2022] PUT /identity/v3/projects/9888d15cc64842d397087e5c5265ca64/users/9b63724faba746b5b4df8bba23c08948/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:21:39.378807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13856a-0fd5-4850-8adf-966a35721756 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:39.379174 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13856a-0fd5-4850-8adf-966a35721756 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:39.379455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13856a-0fd5-4850-8adf-966a35721756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:39.398631 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5b13856a-0fd5-4850-8adf-966a35721756 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:39.399806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5b13856a-0fd5-4850-8adf-966a35721756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:39.458226 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1262/2526] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:39.467942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:39.469723 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:39.524941 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49060986-051b-418f-a56a-6ae394c3ee14 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:39.528507 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:39.529101 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:39.529376 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:39.529644 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:39.539005 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:39.540451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:39.549656 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1265/2527] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:40.763853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8587a-8a10-45bf-b53b-df2d69c1c105 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:40.764360 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8587a-8a10-45bf-b53b-df2d69c1c105 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:40.764459 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8587a-8a10-45bf-b53b-df2d69c1c105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:40.793510 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4fc8587a-8a10-45bf-b53b-df2d69c1c105 None None] MFA Rules not processed for user `9b63724faba746b5b4df8bba23c08948`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:40.794747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4fc8587a-8a10-45bf-b53b-df2d69c1c105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:40.864406 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1263/2528] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:40.875259 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:40.876542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:40.932862 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec52f8-a9cc-48f3-b490-e7d72d10c898 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:40.936547 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:40.937148 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:40.937431 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:40.937703 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:40.946627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:40.949004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:40.950460 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:40.960139 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1266/2529] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:42.693006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2ef7-d2f6-45c5-b225-7331aa6c45c3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:42.693547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2ef7-d2f6-45c5-b225-7331aa6c45c3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:42.693547 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2ef7-d2f6-45c5-b225-7331aa6c45c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:42.712285 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-7f8c2ef7-d2f6-45c5-b225-7331aa6c45c3 None None] MFA Rules not processed for user `9b63724faba746b5b4df8bba23c08948`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:21:42.713468 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-7f8c2ef7-d2f6-45c5-b225-7331aa6c45c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:42.720686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1264/2530] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:21:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:21:44.783953 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:21:44.785426 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:44.854887 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980b8da-30b3-4d59-b94d-54bdc8f4ad1c service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:21:44.860183 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:21:44.860691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:21:44.861006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:21:44.861278 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:21:44.863053 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:21:44.875717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:21:44.877989 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:21:44.888243 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1267/2531] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:21:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:21:52.706884 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:21:54.884553 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=34 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:01.022585 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea9f7e-be0b-482f-b202-a55fac27bc12 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:01.023066 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea9f7e-be0b-482f-b202-a55fac27bc12 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:01.023200 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea9f7e-be0b-482f-b202-a55fac27bc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:01.041476 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-a5ea9f7e-be0b-482f-b202-a55fac27bc12 None None] MFA Rules not processed for user `9b63724faba746b5b4df8bba23c08948`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:01.042646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a5ea9f7e-be0b-482f-b202-a55fac27bc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:01.049327 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1265/2532] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:01.276565 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:01.277975 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:01.289528 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcec2a1-9582-4499-867f-ce0a06201f52 service placement] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:01.293128 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:01.293719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:01.293997 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:01.294271 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:01.295942 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:01.305131 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:01.306530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:01.314398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1268/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:22:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:22:11.036310 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:11.311281 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=6 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:23.586590 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21982541-8767-4e8a-b1a4-b494f25c46dd None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:23.587086 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21982541-8767-4e8a-b1a4-b494f25c46dd None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:23.587190 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21982541-8767-4e8a-b1a4-b494f25c46dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:23.636710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-21982541-8767-4e8a-b1a4-b494f25c46dd None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:23.637890 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-21982541-8767-4e8a-b1a4-b494f25c46dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:23.703990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1266/2534] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:22:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:23.846162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:23.847585 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:23.939264 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c799b-c361-4a69-90f3-970dd3f75618 service neutron] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:23.944905 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:23.945503 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:23.945779 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:23.946050 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:23.954317 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:23.955861 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:23.963591 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1269/2535] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:22:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:22:33.700865 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=40 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:33.960447 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=41 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:41.209666 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:41.210118 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:41.210227 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:41.230135 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:41.231321 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:41.238274 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1267/2536] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 10 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:41.250770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:41.261981 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a09bf-0696-4ac7-9c68-573cbefd33d2 service swift] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:41.266883 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:41.267467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:41.267741 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:41.268160 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:41.276039 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:41.277631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:41.285711 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1270/2537] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:22:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:22:44.273367 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:44.285250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434efcdd-9519-4a7f-9437-9179a568303c None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:44.290428 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:44.291100 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:44.291375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:44.291646 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] PATH_INFO: `/v3/users/9b63724faba746b5b4df8bba23c08948` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:44.292815 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b63724faba746b5b4df8bba23c08948)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:44.294706 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:44.412753 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b63724faba746b5b4df8bba23c08948'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:44.471176 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b63724faba746b5b4df8bba23c08948'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:44.471795 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b63724faba746b5b4df8bba23c08948'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:44.472447 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b63724faba746b5b4df8bba23c08948'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:44.484891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-253fe5e9-2939-4664-8cf0-86482f218af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b63724faba746b5b4df8bba23c08948'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:44.498215 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1268/2538] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:22:44 2022] DELETE /identity/v3/users/9b63724faba746b5b4df8bba23c08948 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:22:44.781321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:44.791960 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a65e5-118e-47d1-a248-d8366ec74943 service swift] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:44.850691 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:44.851314 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:44.851594 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:44.851881 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] PATH_INFO: `/v3/projects/9888d15cc64842d397087e5c5265ca64` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:44.853102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9888d15cc64842d397087e5c5265ca64)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:44.855009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:44.946499 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9888d15cc64842d397087e5c5265ca64', 'request_id': 'req-1c013157-2ad2-4b99-863d-8ee43bbab5b7'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:45.002578 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9888d15cc64842d397087e5c5265ca64', 'request_id': 'req-1c013157-2ad2-4b99-863d-8ee43bbab5b7'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:45.008918 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9888d15cc64842d397087e5c5265ca64', 'request_id': 'req-1c013157-2ad2-4b99-863d-8ee43bbab5b7'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:45.019885 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] The token cache is being invalidate because project 9888d15cc64842d397087e5c5265ca64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:22:45.020205 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:45.020835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-1c013157-2ad2-4b99-863d-8ee43bbab5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:45.022336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1271/2539] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:22:44 2022] DELETE /identity/v3/projects/9888d15cc64842d397087e5c5265ca64 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:22:45.027290 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.027581 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.027853 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.046848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:45.048052 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:45.105961 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1269/2540] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:45.114964 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.115270 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.115538 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.135461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:45.136673 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:45.144393 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1272/2541] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:45.148626 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.158249 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf84ef-6e11-4f29-b1b9-82cd3bb4ba11 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.162919 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.163543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.163820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.164123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.164767 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:22:45.165073 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.166713 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.178608 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1270/2542] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:22:45.182697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.191546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2f10c-544f-4e0d-98b4-89289ed979c9 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.196725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.197307 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.197583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.197850 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.198461 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.200344 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.289721 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1273/2543] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:22:45 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:22:45.293697 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.304087 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9bb76-dcee-4981-98c2-ce9725c0a304 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.350878 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:45.359354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.361475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.361475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.361475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.361475 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.362986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.436649 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1271/2544] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:22:45 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:22:45.440938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.451283 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804416-4c7f-4cc5-a8a5-ed05a0ce47b2 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.456438 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.457024 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.457299 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.457617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.458298 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.460006 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.469764 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1274/2545] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:22:45.474241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.483986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a2e4a-bf34-4d67-8cef-982af88d07d6 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.488661 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401284b-9c04-4aa6-92f5-fe5ce41832d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.491425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401284b-9c04-4aa6-92f5-fe5ce41832d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.491425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401284b-9c04-4aa6-92f5-fe5ce41832d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.491425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401284b-9c04-4aa6-92f5-fe5ce41832d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.491425 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401284b-9c04-4aa6-92f5-fe5ce41832d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.492677 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401284b-9c04-4aa6-92f5-fe5ce41832d4 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.503202 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1272/2546] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 17:22:45.508065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.517910 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2f9f6-d35b-46a8-9b26-b9c8c034038f admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.522416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.523056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.523336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.523612 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] PATH_INFO: `/v3/projects/248c5880aee74b57af619fbc7f13e18c/users/d3dcd31c32b54d169eb0886aad906f93/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.538345 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=248c5880aee74b57af619fbc7f13e18c, user_id=d3dcd31c32b54d169eb0886aad906f93, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.541069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.624921 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1275/2547] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:22:45 2022] PUT /identity/v3/projects/248c5880aee74b57af619fbc7f13e18c/users/d3dcd31c32b54d169eb0886aad906f93/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:22:45.635011 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.635343 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.635614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.664783 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] MFA Rules not processed for user `d3dcd31c32b54d169eb0886aad906f93`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:45.666044 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:45.730214 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1273/2548] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:45.741635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:45.743318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:45.758045 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:45.799004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2eda7-558f-4ce0-9ba6-bc2213cda481 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:45.803262 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4120-4744-45a1-98fc-f0832381cf58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:45.803858 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4120-4744-45a1-98fc-f0832381cf58 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:45.804168 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4120-4744-45a1-98fc-f0832381cf58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:45.804439 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4120-4744-45a1-98fc-f0832381cf58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:45.812627 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4120-4744-45a1-98fc-f0832381cf58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:45.814343 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4120-4744-45a1-98fc-f0832381cf58 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:45.824195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1276/2549] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:22:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:22:55.727349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:55.876769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=20 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:57.059742 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:57.119988 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb22331-c540-4568-b9f4-77e1342299c1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:57.124380 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:57.124998 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:57.125273 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:57.125540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] PATH_INFO: `/v3/users/d3dcd31c32b54d169eb0886aad906f93` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:57.126630 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3dcd31c32b54d169eb0886aad906f93)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:57.128543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:57.342624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3dcd31c32b54d169eb0886aad906f93'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:57.402332 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3dcd31c32b54d169eb0886aad906f93'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:57.402880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3dcd31c32b54d169eb0886aad906f93'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:57.403478 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3dcd31c32b54d169eb0886aad906f93'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:57.416172 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3dcd31c32b54d169eb0886aad906f93'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:57.428383 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1274/2550] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:22:57 2022] DELETE /identity/v3/users/d3dcd31c32b54d169eb0886aad906f93 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:22:57.433506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:57.433972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:57.434089 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:57.452713 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:57.453920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:57.511692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1277/2551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:57.521431 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:57.522829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:57.578248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d89c6-81e5-46d0-924b-d2cb51afd8ae admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:57.582259 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6439b-b628-468b-a81f-e5ac7cbd8afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:57.582891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6439b-b628-468b-a81f-e5ac7cbd8afe service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:57.583166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6439b-b628-468b-a81f-e5ac7cbd8afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:57.583436 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6439b-b628-468b-a81f-e5ac7cbd8afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:57.592842 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6439b-b628-468b-a81f-e5ac7cbd8afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:57.594242 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6439b-b628-468b-a81f-e5ac7cbd8afe service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:57.604035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1275/2552] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:22:58.010623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.012114 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:58.024825 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2701960-c45a-40ef-9699-c1425070c4a4 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.029530 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.030167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.030449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.030725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] PATH_INFO: `/v3/projects/248c5880aee74b57af619fbc7f13e18c` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.031844 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=248c5880aee74b57af619fbc7f13e18c)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.033805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.140042 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248c5880aee74b57af619fbc7f13e18c', 'request_id': 'req-47e54446-f976-4299-93a6-91d9727cdf71'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:58.197099 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248c5880aee74b57af619fbc7f13e18c', 'request_id': 'req-47e54446-f976-4299-93a6-91d9727cdf71'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:58.203130 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248c5880aee74b57af619fbc7f13e18c', 'request_id': 'req-47e54446-f976-4299-93a6-91d9727cdf71'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:58.215400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] The token cache is being invalidate because project 248c5880aee74b57af619fbc7f13e18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81514) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:22:58.215685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:58.216328 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-47e54446-f976-4299-93a6-91d9727cdf71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:22:58.217797 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1278/2553] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:22:58 2022] DELETE /identity/v3/projects/248c5880aee74b57af619fbc7f13e18c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:22:58.222620 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.223105 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.223222 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.242727 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:58.243926 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:58.261115 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:58.302208 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1276/2554] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:58.310696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.310985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.311266 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.331524 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:58.332751 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:58.340753 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1279/2555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:58.344952 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.354721 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6abc7a-1da9-4fec-93a7-3716a51c4a3a None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.359308 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.361556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.361556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.361556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.361556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81513) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:22:58.361556 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.363059 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.374963 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1277/2556] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 17:22:58.380675 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.390535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3526682-caff-4fcb-9ed3-c614895e29d0 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.395406 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.397203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.397203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.397203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.397203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.399150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.495319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1280/2557] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:22:58 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:22:58.499500 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.509873 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91898d93-20bb-45d6-8c29-bdd7902f4f4d admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.564719 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.565248 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.565530 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.565802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] PATH_INFO: `/v3/users` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.566435 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.568349 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.625112 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1278/2558] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:22:58 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:22:58.629362 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.639639 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2472b-e301-462f-8bf0-1fcbf7088aaf admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.644470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.645118 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.645397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.645668 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.646300 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.647972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.657904 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1281/2559] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:22:58.661816 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.676207 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0872a2-7694-4125-a49e-629213156534 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.681804 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e239d-65c4-4ea9-9eaa-6fb41ef4098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.682404 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e239d-65c4-4ea9-9eaa-6fb41ef4098d admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.682683 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e239d-65c4-4ea9-9eaa-6fb41ef4098d admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.682987 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e239d-65c4-4ea9-9eaa-6fb41ef4098d admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.683624 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e239d-65c4-4ea9-9eaa-6fb41ef4098d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.685229 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e239d-65c4-4ea9-9eaa-6fb41ef4098d admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.695781 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1279/2560] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:22:58.700143 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.710238 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e85bc-1363-4c7e-b32e-6a344400c6cb admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.714806 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.715451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.715724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.715994 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] PATH_INFO: `/v3/projects/b842bf06af7b40c987dbc83f387e7174/users/7aa5a5c47d1e4913b7145c4f18a908f9/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.731648 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b842bf06af7b40c987dbc83f387e7174, user_id=7aa5a5c47d1e4913b7145c4f18a908f9, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.734287 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.791419 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1282/2561] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:22:58 2022] PUT /identity/v3/projects/b842bf06af7b40c987dbc83f387e7174/users/7aa5a5c47d1e4913b7145c4f18a908f9/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 17:22:58.801806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b302a7-daeb-411e-917f-0c131894262c None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.802096 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b302a7-daeb-411e-917f-0c131894262c None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.802375 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b302a7-daeb-411e-917f-0c131894262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.830938 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-32b302a7-daeb-411e-917f-0c131894262c None None] MFA Rules not processed for user `7aa5a5c47d1e4913b7145c4f18a908f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:22:58.832121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-32b302a7-daeb-411e-917f-0c131894262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:58.897114 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1280/2562] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:22:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:22:58.907872 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:22:58.909603 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:22:58.932091 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:22:58.970725 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960557cf-82ce-41c5-9503-ce5cc70f4f36 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:22:58.974621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff532d7c1bc4c02838873c67fca787f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:22:58.975257 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:22:58.975533 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:22:58.975800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:22:58.985106 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:22:58.986784 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:22:58.996709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1283/2563] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:22:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:23:00.555049 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:00.555684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:00.555684 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:00.574366 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] MFA Rules not processed for user `7aa5a5c47d1e4913b7145c4f18a908f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:00.575543 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:00.583344 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1281/2564] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:09.048047 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=17 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:10.568940 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:12.836177 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:12.837587 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:12.893754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf3c11-95af-4675-b0ec-4fa2473a0c15 service cinder] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:12.897323 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:12.897919 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:12.898195 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:12.898466 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:12.900166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:12.968824 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:12.970150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138ff7c-23ba-4f22-8680-5f9ce68811e8 service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:12.980852 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1284/2565] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:23:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:23:14.669689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:14.679325 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4eb816-80bc-41e1-8c04-231063f94be3 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:14.682953 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:14.683587 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:14.683870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:14.684170 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:14.685947 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:14.699358 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:14.702802 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:14.716923 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1282/2566] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:23:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:23:18.349755 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b0b09-6275-4034-a335-73c69d8965b9 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:18.350256 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b0b09-6275-4034-a335-73c69d8965b9 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:18.350358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b0b09-6275-4034-a335-73c69d8965b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:18.369972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-280b0b09-6275-4034-a335-73c69d8965b9 None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:18.371398 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-280b0b09-6275-4034-a335-73c69d8965b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:18.425517 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1285/2567] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:23:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:18.437443 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:18.438870 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:18.507093 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1a133-d2ce-477b-a479-dc4354e9dde4 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:18.512180 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:18.513284 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:18.513561 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:18.513829 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:18.521696 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:18.523256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:18.531929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1283/2568] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:23:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:23:19.342654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63538a8f-ad4f-4d34-9876-228c81014bbe None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:19.343016 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63538a8f-ad4f-4d34-9876-228c81014bbe None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:19.343318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63538a8f-ad4f-4d34-9876-228c81014bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:19.362805 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-63538a8f-ad4f-4d34-9876-228c81014bbe None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:19.363992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-63538a8f-ad4f-4d34-9876-228c81014bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:19.371163 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1286/2569] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 10 17:23:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:19.383871 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:19.394205 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268ee3c-7759-424d-a82b-dbe5b74e2f8d service swift] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:19.399269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:19.399955 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:19.400266 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:19.400546 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:19.408708 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:19.410262 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:19.418083 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1284/2570] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:23:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:23:19.943895 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e4d2-70dc-4a13-a382-b44c8144d1af None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:19.944221 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e4d2-70dc-4a13-a382-b44c8144d1af None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:19.944497 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e4d2-70dc-4a13-a382-b44c8144d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:19.964155 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-9e84e4d2-70dc-4a13-a382-b44c8144d1af None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:19.965280 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9e84e4d2-70dc-4a13-a382-b44c8144d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:19.973129 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1287/2571] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 10 17:23:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:19.984722 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:19.995252 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55ac0e-03c2-4672-9049-fa63de542580 service swift] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:20.000296 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:20.000886 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:20.001166 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:20.001440 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:20.009256 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:20.011710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:20.019163 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1285/2572] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:23:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 17:23:20.693749 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:20.694227 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:20.694332 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:20.713218 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] MFA Rules not processed for user `c5b6addb805b4d0a89ee2cd9ac3de27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:20.714399 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:20.721525 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1288/2573] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 10 17:23:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:20.735462 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:20.746243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adf021-5288-4498-a76f-3abd8e3855d6 service swift] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:20.751911 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8facf3-2cff-435e-b3e3-54ac5c9af098 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e348fd8e2b4203b7bc43240cb79d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:20.752557 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8facf3-2cff-435e-b3e3-54ac5c9af098 service swift] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:20.752838 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8facf3-2cff-435e-b3e3-54ac5c9af098 service swift] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:20.753111 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8facf3-2cff-435e-b3e3-54ac5c9af098 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:20.760929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8facf3-2cff-435e-b3e3-54ac5c9af098 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:20.762463 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8facf3-2cff-435e-b3e3-54ac5c9af098 service swift] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:20.770657 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1286/2574] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 10 17:23:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 17:23:23.234583 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:23.263617 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=100 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:23.297236 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161d095-bcfb-4827-86c9-5a2bfdf51adf None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:23.301809 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:23.302444 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:23.302724 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:23.302998 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] PATH_INFO: `/v3/users/7aa5a5c47d1e4913b7145c4f18a908f9` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:23.304154 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa5a5c47d1e4913b7145c4f18a908f9)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:23.306084 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:23.455682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa5a5c47d1e4913b7145c4f18a908f9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:23.456342 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa5a5c47d1e4913b7145c4f18a908f9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:23.522261 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa5a5c47d1e4913b7145c4f18a908f9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:23.535597 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa5a5c47d1e4913b7145c4f18a908f9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:23.536254 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa5a5c47d1e4913b7145c4f18a908f9'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:23.548992 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1289/2575] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:23:23 2022] DELETE /identity/v3/users/7aa5a5c47d1e4913b7145c4f18a908f9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:23:23.553453 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:23.553916 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:23.554030 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:23.572841 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:23.574092 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:23.631693 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1287/2576] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:23:23.641794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:23.643193 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:23.702074 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebefdee-48a1-475a-a1f5-795dbf373cba admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:23.705967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7a77-2aca-4e2c-9a92-84e572041fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:23.706560 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7a77-2aca-4e2c-9a92-84e572041fde service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:23.706833 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7a77-2aca-4e2c-9a92-84e572041fde service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:23.707102 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7a77-2aca-4e2c-9a92-84e572041fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:23.715744 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7a77-2aca-4e2c-9a92-84e572041fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:23.717817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7a77-2aca-4e2c-9a92-84e572041fde service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:23.726920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1290/2577] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:23:23.929245 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:23.930686 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:23.943121 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f731-78f8-4ba7-a46f-a7e6577b8ee8 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:23.948036 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:23.948690 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:23.948990 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:23.949250 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] PATH_INFO: `/v3/projects/b842bf06af7b40c987dbc83f387e7174` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:23.950376 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b842bf06af7b40c987dbc83f387e7174)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:23.952946 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.042095 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:24.053714 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b842bf06af7b40c987dbc83f387e7174', 'request_id': 'req-eba1b982-c351-43ab-a729-700441059485'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:24.059283 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b842bf06af7b40c987dbc83f387e7174', 'request_id': 'req-eba1b982-c351-43ab-a729-700441059485'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:24.066123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b842bf06af7b40c987dbc83f387e7174', 'request_id': 'req-eba1b982-c351-43ab-a729-700441059485'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:24.167238 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] The token cache is being invalidate because project b842bf06af7b40c987dbc83f387e7174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:23:24.167525 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:24.168162 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-eba1b982-c351-43ab-a729-700441059485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:23:24.169627 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1288/2578] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:23:23 2022] DELETE /identity/v3/projects/b842bf06af7b40c987dbc83f387e7174 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:23:24.174634 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.178457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.178457 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.195786 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:24.197030 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:24.241347 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:24.256586 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1291/2579] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:24.264782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.265070 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.265346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.284787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:24.285986 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:24.293929 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1289/2580] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:24.298073 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.307529 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e5198-80fb-415b-85fb-9be9717c7fa3 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.312709 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.313303 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.313575 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.313842 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] PATH_INFO: `/v3/domains` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.314477 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81514) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 17:23:24.314808 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.316543 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.328952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1292/2581] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 10 17:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 17:23:24.333401 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.342361 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580b581-082e-46fd-b711-be7dd08095b9 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.346887 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.347471 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.347745 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.348009 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.348655 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.350507 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.477670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1290/2582] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:23:24 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 17:23:24.481967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.492865 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc744-7f8d-4617-9f4a-2c1c6a505131 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.550479 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.551078 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.551353 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.551623 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.552290 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.554162 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.636793 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1293/2583] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:23:24 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:23:24.640775 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.651286 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38e85f-325d-4880-bc0d-02b21d4c1a28 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.656353 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.656994 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.657269 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.657540 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.658178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.659747 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.670005 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1291/2584] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:23:24.674566 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.684470 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ab4a-5a79-4331-ae8d-7fbca174e725 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.689743 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdfc57-93b3-4d13-a503-a077f2609e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.690336 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdfc57-93b3-4d13-a503-a077f2609e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.690620 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdfc57-93b3-4d13-a503-a077f2609e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.690899 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdfc57-93b3-4d13-a503-a077f2609e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.691534 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdfc57-93b3-4d13-a503-a077f2609e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.693196 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdfc57-93b3-4d13-a503-a077f2609e23 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.703630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1294/2585] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:23:24.707944 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.720396 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227177d-17f1-41af-9c0e-85ffa466a017 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.726850 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.727363 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.727647 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.727975 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] PATH_INFO: `/v3/projects/be063f15b64a45598299aacd0a40e9e8/users/b6576097fd29427dbada288b767be922/roles/3a658e34b7d14750a0169f576d87baef` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.743665 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be063f15b64a45598299aacd0a40e9e8, user_id=b6576097fd29427dbada288b767be922, role_id=3a658e34b7d14750a0169f576d87baef)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.746318 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.807455 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1292/2586] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:23:24 2022] PUT /identity/v3/projects/be063f15b64a45598299aacd0a40e9e8/users/b6576097fd29427dbada288b767be922/roles/3a658e34b7d14750a0169f576d87baef => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:23:24.812500 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f496e2-e4d3-49f5-a63e-718abbdd9a29 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.812787 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f496e2-e4d3-49f5-a63e-718abbdd9a29 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.813061 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f496e2-e4d3-49f5-a63e-718abbdd9a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.831938 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-c6f496e2-e4d3-49f5-a63e-718abbdd9a29 None None] MFA Rules not processed for user `6f35dce3202646389d8bcfb32df2d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:24.833172 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-c6f496e2-e4d3-49f5-a63e-718abbdd9a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:24.897318 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1295/2587] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:24.907524 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:24.910551 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:24.965062 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3544d-92cb-407d-ad88-88ac92842457 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:24.969225 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:24.969823 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:24.970102 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:24.970369 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:24.979178 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:24.980566 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:24.990335 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1293/2588] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:23:29.837923 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:29.838400 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:29.838682 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:29.871125 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] MFA Rules not processed for user `b6576097fd29427dbada288b767be922`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:29.872476 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:29.919739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:29.940658 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1296/2589] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:29.945150 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:29.956393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ff904-c520-474f-9068-33d407a22003 service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:29.961393 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:29.965386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:29.965386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:29.965386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:29.965386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:29.965386 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:30.052906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1294/2590] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 10 17:23:29 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:23:30.053860 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:30.065537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72219-990c-45fc-9ba8-f211ee90080b None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:30.122800 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:30.123948 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:30.124255 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:30.124526 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:30.125167 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:30.127070 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:30.197286 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1297/2591] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 10 17:23:30 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 17:23:30.201140 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:30.212355 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141414b-eb84-4fd6-b29e-f1f53009adf2 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:30.217035 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:30.217625 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:30.218006 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:30.218278 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:30.218909 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:30.220520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:30.231217 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1295/2592] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:23:30.235467 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:30.245288 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682e8e9-eba4-46fd-80a3-49394a2dc0b3 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:30.250432 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:30.251075 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:30.251349 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:30.251621 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] PATH_INFO: `/v3/projects/bb3df00ade3c435f98d806ceec12af83/users/b17fb2fdae3b429d9ed3d0cc03c1713f/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:30.266546 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3df00ade3c435f98d806ceec12af83, user_id=b17fb2fdae3b429d9ed3d0cc03c1713f, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:30.269184 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:30.321972 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1298/2593] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:23:30 2022] PUT /identity/v3/projects/bb3df00ade3c435f98d806ceec12af83/users/b17fb2fdae3b429d9ed3d0cc03c1713f/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:23:30.325891 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:30.336606 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb8c8b-04ef-44a8-8a0b-506124e53d7a admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:30.360978 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:30.395592 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3bc20-3b9f-44b5-aa04-9293abb26061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:30.396241 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bc20-3b9f-44b5-aa04-9293abb26061 admin admin] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:30.396520 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bc20-3b9f-44b5-aa04-9293abb26061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:30.396792 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bc20-3b9f-44b5-aa04-9293abb26061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:30.397430 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bc20-3b9f-44b5-aa04-9293abb26061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:30.399017 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bc20-3b9f-44b5-aa04-9293abb26061 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:30.408807 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1296/2594] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 10 17:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 17:23:30.412959 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:30.423985 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37377aa4-1d3c-4756-9e18-eaa9c3008190 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:30.428663 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:30.429311 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:30.429739 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:30.430144 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] PATH_INFO: `/v3/projects/bb3df00ade3c435f98d806ceec12af83/users/b17fb2fdae3b429d9ed3d0cc03c1713f/roles/6e700d1e090747d6acdf236f6761cbea` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:30.432179 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3df00ade3c435f98d806ceec12af83, user_id=b17fb2fdae3b429d9ed3d0cc03c1713f, role_id=6e700d1e090747d6acdf236f6761cbea)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:30.434789 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:30.445886 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1299/2595] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 10 17:23:30 2022] PUT /identity/v3/projects/bb3df00ade3c435f98d806ceec12af83/users/b17fb2fdae3b429d9ed3d0cc03c1713f/roles/6e700d1e090747d6acdf236f6761cbea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:23:35.815346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb74fdd-4f76-47d4-be7f-3083722bc9d5 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:35.815346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb74fdd-4f76-47d4-be7f-3083722bc9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:35.815346 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb74fdd-4f76-47d4-be7f-3083722bc9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:35.850732 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-8cb74fdd-4f76-47d4-be7f-3083722bc9d5 None None] MFA Rules not processed for user `b17fb2fdae3b429d9ed3d0cc03c1713f`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:35.851668 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-8cb74fdd-4f76-47d4-be7f-3083722bc9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:35.945811 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1297/2596] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:35.962952 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:23:35.964794 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:36.040176 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09cdd5-b7d6-4a40-8ef7-e94e927edfe0 admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:23:36.045654 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:23:36.046267 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:36.046548 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:36.046820 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:36.080817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:36.104110 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:23:36.105697 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:23:36.114508 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1300/2597] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:23:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:23:38.487571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:23:38.488640 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:23:38.488918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:23:38.517634 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] MFA Rules not processed for user `b6576097fd29427dbada288b767be922`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:23:38.518830 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:23:38.528169 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1298/2598] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 10 17:23:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:23:46.122033 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=12 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:23:48.511872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=54 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:24:00.702573 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:24:00.704338 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:24:00.899854 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754531d-054b-42bc-b182-4e0644365755 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:24:00.903504 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:24:00.904147 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:24:00.904425 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:24:00.904692 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:24:00.906397 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:24:00.916363 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:24:00.917756 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303aece-bb17-4de9-a7b9-9569b8b2a48e service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:24:00.926233 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1301/2599] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:24:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:24:10.921977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=26 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:24:16.300710 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:24:16.310131 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd366ce-beef-4377-b9ca-45668df8fb86 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:24:16.313903 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6d51f-5505-4d92-b67a-4b57c69df6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:24:16.314571 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6d51f-5505-4d92-b67a-4b57c69df6bd service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:24:16.314855 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6d51f-5505-4d92-b67a-4b57c69df6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:24:16.315122 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6d51f-5505-4d92-b67a-4b57c69df6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:24:16.324317 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6d51f-5505-4d92-b67a-4b57c69df6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:24:16.325715 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6d51f-5505-4d92-b67a-4b57c69df6bd service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:24:16.336108 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1299/2600] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:24:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:24:26.331597 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=6 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:26:31.841272 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9412f3-a591-48ae-b385-035633af9b0e None None] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:31.841897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9412f3-a591-48ae-b385-035633af9b0e None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:31.841897 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9412f3-a591-48ae-b385-035633af9b0e None None] PATH_INFO: `/` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:31.843410 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1302/2601] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:26:31.850079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f84ff8-b4b7-4be3-9d62-372754861485 None None] REQUEST_METHOD: `POST` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:31.850568 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f84ff8-b4b7-4be3-9d62-372754861485 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:31.850682 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f84ff8-b4b7-4be3-9d62-372754861485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:31.935026 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.auth.core [None req-c8f84ff8-b4b7-4be3-9d62-372754861485 None None] MFA Rules not processed for user `837a59258e6c43c8befebc2639ab16f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81513) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:26:31.936261 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-c8f84ff8-b4b7-4be3-9d62-372754861485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:26:31.969678 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1300/2602] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:26:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 17:26:31.982056 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9412f3-a591-48ae-b385-035633af9b0e None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:26:31.992717 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9412f3-a591-48ae-b385-035633af9b0e None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:26:31.996169 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19d342-1616-4e24-b525-a53304cb90ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:26:31.996754 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d342-1616-4e24-b525-a53304cb90ac service neutron] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:31.997022 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d342-1616-4e24-b525-a53304cb90ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:31.997284 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19d342-1616-4e24-b525-a53304cb90ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:32.004018 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19d342-1616-4e24-b525-a53304cb90ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:26:32.005951 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19d342-1616-4e24-b525-a53304cb90ac service neutron] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:26:32.013641 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1303/2603] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:26:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:26:32.099848 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed7a6f-5305-4b56-ae6d-ef49ebb8cdb1 None None] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:32.100243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed7a6f-5305-4b56-ae6d-ef49ebb8cdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:32.100522 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed7a6f-5305-4b56-ae6d-ef49ebb8cdb1 None None] PATH_INFO: `/` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:32.101872 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1301/2604] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 10 17:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 17:26:32.107269 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] REQUEST_METHOD: `POST` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:32.107580 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:32.107856 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:32.146008 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.auth.core [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] MFA Rules not processed for user `54179215880d4ed291d18944a2f86fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81514) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 17:26:32.147199 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:26:32.177401 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1304/2605] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 10 17:26:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 17:26:32.187979 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed7a6f-5305-4b56-ae6d-ef49ebb8cdb1 None None] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:26:32.199670 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed7a6f-5305-4b56-ae6d-ef49ebb8cdb1 None None] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:26:32.203186 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837a59258e6c43c8befebc2639ab16f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:26:32.203782 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:32.204084 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:32.204354 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:32.215371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:26:32.216824 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:26:32.227906 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1302/2606] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:26:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:26:32.265014 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:26:32.266637 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:26:32.281359 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b70f3-0a37-49fa-ab74-4fefb6215d08 None None] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:26:32.289140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:26:32.289140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:32.289140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:32.289140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:32.290814 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:26:32.291065 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:26:32.292232 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:26:32.304844 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:26:32.306403 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:26:32.318532 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1305/2607] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:26:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 17:26:32.367806 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d871-aa88-4d8e-91a5-1fdd762b1b0f service neutron] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:26:32.373079 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:26:32.373689 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:26:32.373968 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:26:32.374235 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:26:32.375951 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:26:32.388243 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:26:32.389787 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:26:32.400340 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1303/2608] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:26:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:26:42.313920 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=30 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:26:42.395868 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=77 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:27:28.883542 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:27:28.904203 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731653a-744d-4e51-b659-0539b08b67a7 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:27:28.909197 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:27:28.909799 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:27:28.910076 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:27:28.910346 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:27:28.912113 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:27:28.935736 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:27:28.937321 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:27:28.947769 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1306/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:27:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:27:38.943490 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=15 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:28:06.051642 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:06.076593 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb672c-a12a-43a5-8ecd-c2deb56f9d8b service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:06.082201 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:06.083313 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:06.083656 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:06.083992 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:06.086075 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:28:06.100559 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:06.102154 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:06.114371 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1304/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:28:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:28:08.971449 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:08.984013 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5cfab-eded-4476-b02d-7a385ea803fa service placement] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:08.989536 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54179215880d4ed291d18944a2f86fdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:08.990140 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:08.990416 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:08.990685 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:08.992487 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:28:09.004770 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:09.006835 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:09.030475 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1307/2611] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:28:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:28:14.930660 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:14.943281 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30702b6-fa31-4522-9cfb-51ac41d92ea8 service nova] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:14.948880 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:14.949688 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] REQUEST_METHOD: `GET` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:14.949967 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:14.950298 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:14.952107 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:28:14.965924 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:14.967614 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:14.978962 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1305/2612] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:28:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:28:18.349166 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:18.431333 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2df1-5341-4ceb-b6ef-292c242b2e03 service nova] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:18.435876 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:18.436535 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:18.436814 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:18.437085 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] PATH_INFO: `/v3/users/b6576097fd29427dbada288b767be922` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:18.438189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6576097fd29427dbada288b767be922)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:18.440150 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:18.613817 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6576097fd29427dbada288b767be922'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:18.614970 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6576097fd29427dbada288b767be922'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:18.686189 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6576097fd29427dbada288b767be922'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:18.698630 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6576097fd29427dbada288b767be922'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:18.699275 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6576097fd29427dbada288b767be922'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:18.711720 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1308/2613] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:28:18 2022] DELETE /identity/v3/users/b6576097fd29427dbada288b767be922 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:28:18.943621 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:18.963930 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afc900-f78f-410a-b8ac-6f15d02582b1 service placement] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:19.035901 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:19.036575 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:19.036856 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:19.037132 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] PATH_INFO: `/v3/projects/be063f15b64a45598299aacd0a40e9e8` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:19.041918 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be063f15b64a45598299aacd0a40e9e8)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:19.043818 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:19.155377 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be063f15b64a45598299aacd0a40e9e8', 'request_id': 'req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.160941 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be063f15b64a45598299aacd0a40e9e8', 'request_id': 'req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.167123 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be063f15b64a45598299aacd0a40e9e8', 'request_id': 'req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.245934 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] The token cache is being invalidate because project be063f15b64a45598299aacd0a40e9e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:28:19.246156 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.246773 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.248268 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1306/2614] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:28:18 2022] DELETE /identity/v3/projects/be063f15b64a45598299aacd0a40e9e8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:28:19.252319 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:19.253686 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:28:19.267750 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b07b85-aef5-4eb6-a32f-5e96eb17978d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:19.310367 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:28:19.322451 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:19.323080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:19.323358 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:19.323631 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] PATH_INFO: `/v3/users/b17fb2fdae3b429d9ed3d0cc03c1713f` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:19.324783 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17fb2fdae3b429d9ed3d0cc03c1713f)` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:19.326719 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:19.511696 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17fb2fdae3b429d9ed3d0cc03c1713f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.512361 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17fb2fdae3b429d9ed3d0cc03c1713f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.588999 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17fb2fdae3b429d9ed3d0cc03c1713f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.601004 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17fb2fdae3b429d9ed3d0cc03c1713f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.601635 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.notifications [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17fb2fdae3b429d9ed3d0cc03c1713f'} {{(pid=81514) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:19.613485 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1309/2615] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 10 17:28:19 2022] DELETE /identity/v3/users/b17fb2fdae3b429d9ed3d0cc03c1713f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:28:19.862898 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Authenticating user token {{(pid=81513) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:28:19.864772 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.fernet_utils [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81513) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:28:19.878695 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0a2b-ca7a-4953-87ed-800f0d6b09f1 admin admin] Validating token access rules against request {{(pid=81513) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:28:19.923501 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:28:19.936553 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f35dce3202646389d8bcfb32df2d4b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3acc8f5bcdf4b42bd810be11a69d5c4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81513) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:28:19.946002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:28:19.946002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:28:19.946002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] PATH_INFO: `/v3/projects/bb3df00ade3c435f98d806ceec12af83` {{(pid=81513) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:28:19.946002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3df00ade3c435f98d806ceec12af83)` {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:28:19.946002 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] RBAC: Authorization granted {{(pid=81513) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:28:20.031828 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3df00ade3c435f98d806ceec12af83', 'request_id': 'req-64ea528d-31a5-407f-bdaa-f68237e63621'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:20.038081 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3df00ade3c435f98d806ceec12af83', 'request_id': 'req-64ea528d-31a5-407f-bdaa-f68237e63621'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:20.044523 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3df00ade3c435f98d806ceec12af83', 'request_id': 'req-64ea528d-31a5-407f-bdaa-f68237e63621'} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:20.095833 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] The token cache is being invalidate because project bb3df00ade3c435f98d806ceec12af83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81513) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 17:28:20.096157 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:20.096820 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG keystone.notifications [None req-64ea528d-31a5-407f-bdaa-f68237e63621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81513) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 17:28:20.098433 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: [pid: 81513|app: 0|req: 1307/2616] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 10 17:28:19 2022] DELETE /identity/v3/projects/bb3df00ade3c435f98d806ceec12af83 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 17:28:29.610062 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:28:30.092536 dv-cinder-834540-2-iscsi devstack@keystone.service[81513]: DEBUG dbcounter [-] [81513] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81513) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 17:32:31.811007 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Authenticating user token {{(pid=81514) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 17:32:31.813009 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:32:31.882967 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccdd63-d941-4f40-84bc-42c7035a456d admin admin] Validating token access rules against request {{(pid=81514) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 17:32:31.939909 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882630eb293423e84ecbd0c63fb6733', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '450c1373880a4e2ba0eb9fd1c5201580', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81514) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 17:32:31.940537 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] REQUEST_METHOD: `GET` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 17:32:31.940813 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] SCRIPT_NAME: `/identity` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 17:32:31.941080 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81514) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 17:32:31.942772 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.fernet_utils [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81514) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 17:32:31.953506 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 17:32:31.955069 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f398-f1a8-49a2-936d-a57b653187bd service placement] RBAC: Authorization granted {{(pid=81514) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 17:32:31.964977 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: [pid: 81514|app: 0|req: 1310/2617] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 10 17:32:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 17:32:41.960777 dv-cinder-834540-2-iscsi devstack@keystone.service[81514]: DEBUG dbcounter [-] [81514] Writing DB stats keystone:SELECT=58 {{(pid=81514) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}